Post on 14-Jan-2016
description
Assessment Standards Knowledge exchange
Engaging students with assessment feedback
Prof. Margaret Price, Director ASKe Centre for Excellence
FDTL Engaging Students with assessment feedbackhttps://mw.brookes.ac.uk/display/eswaf/Home
ASKe Assessment Standards Knowledge exchangeCentre for Excellence in Teaching and Learning
http://www.brookes.ac.uk/aske/!aske@brookes.ac.uk
Purpose of Workshop
• Problems and responses
• Engagement with feedback
• Where to start
• Resources and effectiveness
We have a problem!
• Surveys and audits
• Research literature
Feedback problems
• Unhelpful feedback (Maclellan, 2001)• Too vague (Higgins, 2002)• Subject to interpretation (Ridsdale, 2003)• Not understood (e.g. Lea and Street, 1998)• Don’t read it (Hounsell, 1987. Gibbs & Simpson
2002)• Damage self-efficacy (Wotjas, 1998)• Has no effect (Fritz et al, 2000)• Seen to be too subjective (Holmes & Smith,
2003)
Some responses to the feedback ‘crisis’:
• Provide more of the same • Simplistic rules about timing• Standardisation• Label feedback
• Setting expectations• Introducing new methods
• a complex problem so no simple solution
Exploring feedback (activity)
• What is its purpose?
• What counts as feedback?
• What can it achieve?
• How do you know it is working?
Student engagement with feedback
Assessment feedback
CollectionImmediate attention
Cognitive response
Immediate or latent action
Specific, immediate action, such as applying feedback insights to next
assignment
Developmental changes which generate the latent potential for future action
Self-efficacy
ConfidenceResources; skills
Opportunities for action
... or, after reflection, rejects, or
‘mis’understands, etc.
… or does not collect
... or cannot understand, so
does not proceed
.. or cannot read, so ignores
OU
TC
OM
E
Student outcome influences later
engagement with feedback
Price et al (submitted)
Activity
In 3’s, discuss:
• How do you currently prepare students to understand and engage with feedback?
Where to start
Preparation and setting expectations early in the programme
Identifying ‘feedback moments’ and application opportunities within the programme
Emphasize the relational dimension of feedback
Building in space for dialogue
What can we do? (1)
• Aligning expectations (of staff & students, & between teams of markers)
• Identify what is feasible in a given assessment context - written feedback can often do little more than ‘diagnose’ development issues and then direct students to other resources for help and support
• Identifying all feedback available
• Ensure it is timely - ‘quick and dirty’ generic feedback, feedback on a draft, MCQs & quizzes, etc. (using technology may help)
• Model and encourage the application of feedback
What can we do? (2)
• Require and provide feedback on self-assessment
• Improve the linkage of assessment strategies across programmes and between modules/units
• Consider the role of marks - they obscure feedback
• Reduce over-emphasis on written feedback - oral can be more effective (McCune, 2004). Face to face feedback with 140 students (FDTL Case study: https://mw.brookes.ac.uk/display/eswaf/Home.
• Review resource allocations
What can we do (3)
• Support the relational dimension of feedback Students say that relationships in which staff are supportive and approachable help them to engage Avoid anonymous marking
Ensure associate (and permanent) staff have sufficient time and/or space
Provide some continuity of staff contact (personal tutors)
Provide opportunity for dialogue (e.g. discuss feedback in class, peer review, peer assisted learning)
Peer marking using model answers (Forbes & Spence, 1991)
Scenario:• Engineering students had weekly maths problem sheets
marked and problem classes• Increased student numbers meant marking impossible
and problem classes big enough to hide in• Students stopped doing problems• Exam marks declined (Average 55%>45%)Solution:• Course requirement to complete 50 problem sheets• Peer assessed at six lecture sessions but marks do not
count• Exams and teaching unchangedOutcome: Exam marks increased (Av. 45%>80%)
Peer feedback - Geography (Rust, 2001)
Scenario• Geography students did two essays but no apparent improvement
from one to the other despite lots of tutor time writing feedback• Increased student numbers made tutor workload impossible
Solution:• Only one essay but first draft required part way through course• Students read and give each other feedback on their draft essays• Students rewrite the essay in the light of the feedback• In addition to the final draft, students also submit a summary of how
the 2nd draft has been altered from the1st in the light of the feedback
Outcome: Much better essays
Peer feedback - Computing (Zeller, 2000*)
The Praktomat system allows students to read, review, and assess each other’s programs in order to improve quality and style. After a successful submission, the student can retrieve and review a program of some fellow student selected by Praktomat. After the review is complete, the student may obtain reviews and re-submit improved versions of his program. The reviewing process is independent of grading; the risk of plagiarism is narrowed by personalized assignments and automatic testing of submitted programs.
In a survey, more than two thirds of the students affirmed that reading each other’s programs improved their program quality; this is also confirmed by statistical data. An evaluation shows that program readability improved significantly for students that had written or received reviews.
[*Available at: http://www.infosun.fim.unipassau.de/st/papers/iticse2000/iticse2000.pdf]
Figure 1: Peer-review as a method of encouraging students to discuss and compare their understanding of assessment criteria
2. Students bring draft individual
assignments for peer review
MODULE TIMELINE
4. Students rewrite and
submit individual
assignments
Week1 Week12
1. Tutor leads discussion on assessment criteria and
process of peer review
3. In-class discussions
between student groups as they
review each other’s work, monitored by
tutor.
5. Tutor marks
assignments and prepares
feedback
6. Tutor hands back assignments
and leads discussion on
feedback
Assignmentpoint
OU
T O
F C
LA
SS
A
CT
IVIT
YIN
-CLA
SS
A
CT
IVIT
Y
Figure 1
Figure 2: the use of 'exemplars' as amechanism for encouraging dialogue about assessment criteria
1. Tutor leads discussion of previously-
marked exemplars
annotated with feedback
OU
T O
F C
LAS
S
AC
TIV
ITY
IN-C
LAS
S
AC
TIV
ITY
2. Students write and
submit individual
assignments
3. Tutor marks
assignments and prepares
feedback
4. Tutor hands back assignments
and leads discussion on
feedback
MODULE TIMELINE
Assignmentpoint
Week1 Week12
Figure 2
Figure 3: Generic feedback and self critique
1. Students draft and submit
individual assignments
2. Tutor marks sample of
assignments and prepares
generic feedback
5. Tutor grades
assignments
4. Students rewrite and submit individual assignments with
reflective commentary on how they have incorporated the generic feedbackO
UT
OF
CLA
SS
A
CT
IVIT
YIN
-CLA
SS
A
CT
IVIT
Y
MODULE TIMELINE
Assignmentpoint
Week1 Week12
3. In-class discussion of generic cohort
feedback based on
coursework sample
6. Tutor hands back assignments with grade
only
Figure 3
Activity
Individually:
Choose one or more specific ideas to improve feedback that you think you could use. In as much detail as possible, identify how you would put the idea/s into practice.
In pairs:
Take it in turns to explain your plans to your partner. The job for the listener is to be a friendly and constructive critic
Feedback moments
• Where there is a clear opportunity to apply feedback
• Pre assessment
• Reflection points
Identify them within each programme
Figure 4:Taking an overview
Student preparation – HE orientation
PR
OG
RA
MM
E D
UR
AT
ION
End Yr1
End Yr2
End Yr3
Regular review meetings with personal tutor to
discuss feedback
Sem 1: Introduction to self-assessment and peer review.
Discussion of criteria and use of exemplars
Sem 2: Introduction to group work, continued development of peer and self-assessment. Continued
use of exemplars
Year 2: Support in the transition to Stage 2 modules, with more
formative feedback
Year 3: Expectation that students will engage in more self-
assessment, and will demonstrate ability to critique own work
Student involvement in peer-assisted learning
(voluntary)
Student involvement in mentoring others (voluntary)
PROGRAMME LEVEL MODULAR LEVEL
Refs
• Forbes, D., & Spence, J. (1991). An experiment in assessment for a large class. In R.Smith (Ed.), Innovations in engineering education. London: Ellis Horwood.
• Fritz, C.O., Morris, P.E., Bjork, R.A., Gelman, R. & Wickens, T.D. (2000) When further learning fails: Stability and change following repeated presentation of text, British Journal of Psychology, 91, pp. 493-511
• Gibbs, G. & Simpson, C. (2002) Does your assessment support your students’ learning available at: http://www.brookes.ac.uk/services/ocsd/1_ocsld/lunchtime_gibbs.html (accessed November 2002)
• Higgins, R., Hartley, P. & Skelton, A. (2002) The conscientious consumer: reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27 (1) pp. 53-64
• Hounsell, D. 1987. Essay writing and the quality of feedback. In J.T.E. Richardson, M.W. Eysenck & D. Warren-Piper, eds. Student Learning: Research in Education and Cognitive Psychology, 42, no.2: 239-54.
• Holmes, L. E., & Smith, L. J. (2003). Student evaluations of faculty grading methods. Journal of Education for Business, Vol. 78 No. 6, 318.
• Lea, M. & Street, B. (1998) Student Writing in Higher Education: an academic literacies approach. Studies in Higher Education, 23 (2), pp. 157-172
• McCune, V., (2004) Development of first –year students’ conceptions of essay writing. Higher Education, 47, pp. 257-282.
• Maclellan, E. 2001. Assessment for learning, the different perceptions of tutors and students. Assessment and Evaluation in Higher Education. 26, no.4: 307-318
• Ridsdale, M.L.“I’ve read his comments but I don’t know how to do”:International postgraduate student perceptions of written supervisor feedback. In ‘Sources of confusion: refereed proceedings of the national language and academic skills conference held at La Trobe University, November 27-28,2000’ edited by \k \charnock, pp272-282.
• Rust, C. (2001) A briefing on assessment of large groups, LTSN Generic Centre Assessment Series, No. 12, York, LTSN
• Wotjas, O. 1998. Feedback? No, just give us the answers. Times Higher Education Supplement. September 25