Assessment and Feedback PrinciplesTheory and Practice

David NicolVisiting Professor in Centre for Higher Education

PracticeUniversity of Ulster

Emeritus Professor: University of StrathclydeWebsite: reap.ac.uk

Festival of Innovative Practice 18 June 2013Magee Campus

Plan

Background: Assessment and Feedback principles

Re-engineering Assessment Practices (REAP) project: University of Strathclyde 2005-7

Ulster’s principles: research basis and examples Dynamics of Implementation JISC-funded Viewpoints project (University of

Ulster): the A&F design toolkit A principles-based discourse approach to

institutional change.

REAP: Re-engineering Assessment Practices project Scottish Funding Council (£1m): 2005-2007

Goals: learning quality and teaching efficiencies 3 HEIs (Strathclyde, Glasgow Caledonian Business

School, Glasgow University) Large 1st year classes (160-900 students) Range of disciplines (19 modules ~6000

students) Many technologies: online tests, simulations,

discussion boards, e-portfolios, e-voting, peer/feedback software, VLE, online-offline

Outputs: new practices, institutional embedding Assessment for learner self-regulationwww.reap.ac.uk

Barriers to transformational change in teaching and learning institution-

wide1. Lack of shared educational frame of reference to

guide innovations in practice2. Difficulty defining good educational practice3. Isolation of academics from educational research4. Disciplinary differences in teaching and learning5. Weak links between local practices and

educational policies and strategies6. Challenge of getting multi-stakeholder buy-in

See Nicol & Draper (2009)

Background (1)

Gibbs, G. & Simpson, C (2004)Conditions under which assessment supports students learning, Learning and Teaching in Higher Education, 1, 3-31.

See: Formative Assessment in Science Teaching (FAST)

project at: http://www.open.ac.uk/science/fdtl/

Gibbs and Simpson (2004)

Assessment tasks [Conditions 1-4]1. Capture enough study time (in and out of class)2. Are spread out evenly across timeline of study3. Lead to productive activity (deep vs surface)4. Communicate clear and high expectations

i.e concern here is with ‘steers’ about how much work to do and the quality of what is done

Background (2)Literature Review Nicol, D. & Macfarlane-Dick, D. (2006). Formative

assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 34 (1), 199-218

Nicol, D & Milligan, C. (2006), Rethinking technology-supported assessment practices in relation to the seven principles of good feedback practice. In C. Bryan & K. Clegg, Innovative assessment in higher education, Routledge.

Background Student Enhanced Learning through Effective

Feedback [SENLEF] project funded by HE Academy

Key aspects of research Students are always engaged in self-assessment/

self-regulation of their own learning (Winne, 2005).

The act of using teacher feedback implies that self-assessment must be present (Black and Wiliam, 1998).

Feedback in HE is being reduced so how are students still learning.

The question is: how can we scaffold students’ learning so they become better at self-regulation (Lajoie, 2005)

Rethinking assessment and feedback

1. Consider self and peers as much as the teacher as sources of assessment and feedback

Tap into different qualities than teacher can provide

Provides considerable learning benefits Better use of teacher time2. Focus on every step of the cycle:

Understanding the task criteria (Sadler, 1983) Applying what is learned in action

3. Not just written feedback: Also computer, dialogue, formal and informal

Seven principles of good A&F practice1. Clarify what good performance is (goals, criteria,

standards).2. Facilitate the development of reflection and self-

assessment in learning 3. Deliver high quality feedback to students: that

enables them to self-correct4. Encourage peer and student-teacher and peer

dialogue around learning5. Encourage positive motivational beliefs & self

esteem through assessment6. Provide opportunities to act on feedback7. Provide information to teachers that can be used to

help shape their teachingNicol and Macfarlane-Dick (2006)

Two meta-principles

Meta-principle 1: time and effort on task (in and out of class (engagement) i.e. steers on how much work to do and when – Gibbs and Simpson 4 conditions

Meta-principle 2: developing learner self-regulation (empowerment/self-regulation) i.e steers to encourage ownership of learning – the Nicol and Macfarlane-Dick 7 principles.

Original Principles of Assessment and FeedbackEngagement1. Assessment tasks capture sufficient study time (in and out of class)2. Are spread out evenly across timeline of study3. Lead to productive activity (deep learning)4. Communicate clear and high expectationsGibbs and Simpson (2004) Empowerment5. Clarify what good performance is (goals, criteria, standards).6. Facilitate the development of reflection and self-assessment in

learning 7. Deliver high quality feedback to students: that enables them to self-

correct8. Encourage peer and student-teacher and peer dialogue around

learning9. Encourage positive motivational beliefs & self esteem through

assessment10.Provide opportunities to act on feedback11.Provide information to teachers that can be used to help shape their

teachingNicol and Macfarlane-Dick (2006)

The REAP project

Implementation

Local redesigns 19 module redesigns: principles guided

implementations Success: learning gains in exams (11 out of 19

modules) improved quality w/o extra costs, high levels of student satisfaction, efficiency gains.

Institutional developments Dep. Principal T&L set up working group: new A&F

policy (Strathclyde) grounded in principles agreed by Senate

Principles embedded in QA procedures Many departmental/university initiatives

referencing REAP and using the principles Sharing of good practice using principles as

reference Widespread use of principles:nationally &

internationally

Barriers to transformational change in teaching and learning institution-

wide1. Lack of shared educational frame of reference to

guide innovations in practice2. Difficulty defining good educational practice3. Isolation of academics from educational research4. Disciplinary differences in teaching and learning5. Weak links between local practices and

educational policies and strategies6. Challenge of getting multi-stakeholder buy-in

See Nicol & Draper (2009)

Thinking behind REAP The vision – Assessment and Feedback should

support the development of learner self-regulation A set of assessment & feedback principles based

on research – to operationalise this vision Principles as translation device – to make the

research accessible to busy academics Implementation strategy: REAP team supported

course teams engaged in redesigns (i.e. application of A&F principles in modules and programmes and technologies)

Evaluation of innovations to build institutional commitment

Addressing the barriers to change Lack of shared frame of reference to guide

innovations: the big idea and principles Difficulty defining good educational practice Isolation of academics from educational

research: simplifying research into some key ideas

Disciplinary differences – tight-loose structure of principles

Weak links between local practices and educational policies and strategies – single set of educational ideas, tying each course design to policy framework

Challenge of getting multi-stakeholder buy-in

Good formative assessment and feedback practices should:1. Help clarify what good performance is (goals, criteria, standards)2. Encourage time an effort on challenging learning tasks3. Deliver high quality feedback information that helps learners self-correct4. Provide opportunities to act on feedback 5. Ensure that summative assessment supports formative

learning processes6. Encourage interaction and dialogue around learning (peer, t-st)7. Facilitate the development of self-assessment and reflection in learning8. Give choice in the topic, method, criteria, weighting or timing of

assessments.9. Involve students in decision making about assessment policy and

practice10. Support the development of learning groups and communities11. Encourage positive motivational beliefs and self-esteem12. Provide information to teachers that can be used to help shape teaching

Clarify good performanceHelp to clarify, from the early stages of a programme, what good performance means (goals, criteria, standards);

Encourage time and effort on taskEncourage 'time and effort' on challenging learning tasks, which recognise the importance of learning from the tasks, not just demonstrating learning through tasks;

Deliver timely high quality feedbackDeliver timely learner-related feedback information that helps students to self-correct and communicates clear, high, expectations and professionalism;

Provide opportunities to act on feedbackProvide opportunities for students to act on feedback and close any gap between current and desired performance through complementary and integrated curriculum design and pedagogic practice;

Encourage positive motivational beliefsEnsure that all assessment has a beneficial, constructive, impact on student learning, encouraging positive motivational beliefs, confidence and self-esteem;

Develop self-assessment and reflectionFacilitate the development of self- and peer-assessment skills and reflection on learning, to enable students to progressively take more responsibility for their own learning, and to inspire a lifelong capacity to learn;

Encourage interaction and dialogueEncourage interaction and dialogue around learning and professional practice (student-student, lecturer-student and lecturer-lecturer) including supporting the development of student learning groups and peer learning communities.

Principles of assessment and feedback for learning

Good assessment and feedback practice shouldClarify what good performance is (goals, criteria, standards).

Teacher provides criteria before task insufficient Teachers provide a range of model answers Students derive criteria from exemplars before the

task Students create criteria for new task Students compare own work against standards

descriptors Students rank peer work in order of quality and

discuss

Students must develop a concept of quality

Good assessment and feedback should:Encourage time and effort on task

Reduce size and increase number of learning tasks Give small number of marks for completion Provide tasks as out-of class homework activities

that link to in-class activities (e.g. MCQs) Group tasks are important here (e.g. EVS)

Create structures that promote ownership and effort

Good assessment and feedback practice should:Delver timely high quality feedback: that helps students to self-correct

Students request feedback they wish (cover

sheet) Feedback on processes and skills –

maximise transfer Don’t give feedback – point to resources

where answer/issue can be elaborated Feedback on students’ self-assessments

and/or peer reviews

Calibrate students’ ability to make evaluative judgements

(see Hattie and Timperley, 2007)

Good assessment and feedback should:Provide opportunities to act on (respond to) feedback

Students respond to teacher feedback in

writing Sequence assignments so feedback is used Provide feedback as action points Students say how used feedback when

submit next assignment [proforma] Reward use of feedback in a new task

(Gunn, 2010)

Ensures feedback is processed and leads to knowledge building. Key principle if your goal is to enhance NSS results

Good assessment and feedback should:Encourage positive motivational beliefs and self-esteem

Encourage climate of respect and accountability Sequence tasks for progressive level of difficulty Use real life (authentic) learning tasks Give learners choice in topic, methods, criteria Implement other principles (group working etc) Balance structure with increasing learner

responsibility

About giving students a sense of control over their learning

Good assessment and feedback should:Develop self-assessment (and peer assessment skills) and reflection.

Students identify what is strong and weak when they hand in an assignment

Provide an abstract with an essay (reflection)

Implement peer review where students’ comment on each other’s work (see Nicol, 2013: Nicol et al, 2013)

then review their own work

Give students practice in making evaluative judgements

Good assessment and feedback should:Encourage interaction and dialogue around learning (peer, student-lecturer, lecturer-lecturer)

Electronic voting methods: polling and peer

discussion Discussions of feedback in tutorials Peer review using pairs and groups Collaborative assignments Wrap dialogue around all assessment processes

(Nicol, 2010)

Attenuates teachers’ voice and strengthens students’ voice (shifts responsibility towards students)

Key focus of current research Developing students’ capacity to make

evaluative judgements about own and other’s work (Boud, 2013: Sadler, 2013: Nicol, 2013).

Making judgements has not received enough attention in HE (Cowan, 2010)

Evaluative capability underpins all graduate attributes (Nicol, 2009)

Assessment in HE does not replicate what happens in professional practice

Peer review – producing reviews is qualitatively different from receiving reviews (Nicol et al, 2013)

Questions and discussion

Bigger picture: learning from REAP

Principles-based approach has great potential People talked about and used the principles –

an emerging discourse Widespread take-up of principles in UK and

internationally – principles legacy more enduring than REAP innovations themselves

Question: how to take things further forward so that the ideas and discourse spread more rapidly and deeply across the whole institution.

The JISC-funded Viewpoints project

Took the REAP assessment and feedback principles and put them ‘in the hands of the user’

The principles as a workshop tookit - prompts to think through an assessment and feedback design.

The principles as ‘social objects’ to seed and sustain a new educational discourse.

http://wiki.ulster.ac.uk/display/VPR/Home

Assessment and Feedback Principles

REAP (http://www.reap.ac.uk/)

1. Clarify good performance.

2. Encourage time and effort on task.

3. Deliver high quality feedback.

4. Provide opportunities to act on feedback.

5. Encourage interaction and dialogue.

6. Develop self-assessment and reflection.

7. Give assessment choice.

8. Encourage positive motivational beliefs.

9. Inform and shape your teaching.

*Implementation ideas for each principle on back of cards.

Viewpoints Assessment and Feedback workshop

The Artefacts:Timeline worksheet & Principles

Discussing the objective

Course team agree the objective for their session and write it at the top of the module worksheet.

Reading the front of the cards

The team read the principles on the front of the cards, choosing ones appropriate to their objective.

Mapping the cards to the learner timeline

The team take their selected cards and map them to the appropriate point on the timeline (e.g. at the induction phase, during first few weeks of course)

Reading examples on cards

Workshop participants turn the cards over and read the examples/ideas on the back.

Choosing relevant examples

The team might select or adapt any examples that would fit with their course objective and their teaching practice.

Adding in own ideas/comments

The team produce their own examples, ideas, comments and use post-it notes develop their own assessment and feedback design.

Workshop outputs• Form basis of an

Assessment & Feedback (or other theme) strategy

• Provide reference for future course team discussions (planning resource)

• Key information will be transcribed into table (Word or other format)

These can be orderly or messy – it’s up to you

Users/teams define the outputs

Features of workshop discourse

1. Agenda set by participants (course teams)2. Peer interaction, discussion and sharing of ideas3. Concepts/examples on cards inform discussion (i.e.

It is research informed)4. Principles call on each other in use5. Construct & co-construct meaning – not about

telling6. Learner-focused not content focused7. Problem-focused yet exploratory and creative8. Structured by a timeline9. Socially engaging- like a board game10. Ideas extend beyond the workshop

Evaluation of REAP, Viewpoints and of other HE institutions that have used

educational principles Model of change extrapolated Principles-based discourse model Discourse shapes how people think about things

and therefore how they act; and how people act and think about things shape their discourses. (Marshak and Grant, 2011)

Way of addressing change in complex organisations

Discourse is not just about conversations but also about written texts, official documents, emails, memos, stories, narratives, metaphors, slogans etc.

Focuses on meaning-making and the social construction of reality

About back-stage processes as much as front-stage events

Focus is discourse as a driver for change not just as a symptom

There are already discourses about assessment and feedback but are these educationally informed?

Principles-based discourse model for change

Extending the reach: many contexts of application

Many reference points and opportunities for discussion in other forums and contexts

Facilitated by support services (LLL, TEL, Acad Practice Unit), staff induction, staff development, through revalidation & course review processes, and devolved to departments to meet their needs

Links made when discussing other agendas – graduate attributes, employability etc.

Through student initiatives

Embedding the discourse in documents and texts

In educational policy (Strathclyde, Ulster) In publicity materials for students and staff In reference documents – QAA procedures,

audit, course approval and review, student evaluations, external examiners briefing docs, etc.

As framework to comment on other educational innovations – linking language

Some of my Publications

Nicol, D., Thomson, A. and Breslin, C (2013) Rethinking feedback practices in higher education: a peer review perspective, Assessment and Evaluation in Higher Education, contact DN

Nicol, D (2013), Resituating feedback from the reactive to the proactive. In D. Boud and L. Malloy (Eds) Effective Feedback in Higher and Professional Education: understanding it and doing it well, Routledge UK

Nicol, D (2011) Developing students’ ability to construct feedback, QAA Scotland, Enhancement Themes. Available on Scottish Enhancement Themes website

Nicol, D (2010) The foundation for graduate attributes: developing self-re gulation through self and peer assessment, QAA Scotland, Enhancement Themes.

Available at: Nicol, D (2010) From monologue to dialogue: improving written feedback in mass higher education, Assessment and Evaluation in Higher Education, 35:5, 501-517

Nicol, D and Draper, S (2010), A blueprint for transformational organisational change in HE: REAP as a case study (see reap.ac.uk website)

Nicol, D (2009), Transforming assessment and feedback: Enhancing integration and empowerment in the first year, Published by Quality Assurance Agency, Scotland

Nicol, D (2009), Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3), 335-352

Nicol, D (2007) E-assessment by design: using multiple-choice tests to good effect, Journal of Further and Higher Education.31(1), 53-64..

Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), 199-218.

See also www.reap.ac.uk for copies.