Enhancing assessment and feedback in the first year: principles and practices

David Nicol Professor of Higher Education

Centre for Academic Practice and Learning Enhancement (CAPLE]

Director, REAP and PEER projects (www.reap.ac.uk) University of Strathclyde

University UCD Dublin 20th January 2011

NSS: Assessment and feedback (2008)

No

Survey Statement England

Scotland

Northern Ireland

5. The criteria used in marking have been clear in advance

69 69 69

6. Assessment arrangements and marking have been fair

74 74 73

7. Feedback on my work has been prompt

56 51 53

8. I received detailed comments on my work

61 52 52

9. Feedback on my work has helped clarify things I did not understand

56 51 50

22.

Overall, I am satisfied with the quality of the course

82 86 83

Plan

BackgroundRe-engineering Assessment Practices (REAP) projectConcepts and ideasCase studies of practice from REAPGuidelines for implementationShare ideas/practices

Background

Departments and faculties: REAP supporting local innovations in assessment & feedbackPolicy/strategy: led development of policy (based on REAP principles)Students: ‘Feedback as dialogue’ campaignDocumentation: Course approval and review (JISC funded)HE Sector: Project facilitator for QAA Scotland on A&FResearch: SENLEF, REAP, PEER etc

See www.reap.ac.uk

Re-engineering Assessment Practices project

Scottish Funding Council (£1m)Strathclyde, Glasgow and Glasgow CaledonianLarge 1st year classes (160-900 students)A range of disciplines (19 modules ~6000 students)Many technologies: online tests, simulations, discussion boards, e-portfolios, e-voting, peer/feedback software, VLE, online-offlineLearning quality and teaching efficienciesAssessment for learner self-regulationwww.reap.ac.uk

First Year: The academic experience

What is important in the first year?

Coping with transitionUnderstanding what is requiredEngagement with academic programmesReceiving support and feedback Experiences of successFeeling in control of own learningBelief that you can succeedA sense of belonging within the academic and social culture

Based on research by Yorke (UK) and Tinto (US)

Background (1)

Gibbs, G. & Simpson, C (2004) Conditions under which assessment supports students learning, Learning and Teaching in Higher Education, 1, 3-31.

See:

Formative Assessment in Science Teaching (FAST) project at: http://www.open.ac.uk/science/fdtl/

Gibbs and Simpson (2004)

Assessment tasks [Conditions 1-4]1. Capture sufficient study time (in and out of

class)2. Are spread out evenly across timeline of study3. Lead to productive activity (deep vs surface)4. Communicate clear and high expectations

i.e concern here is with ‘steers’ about how much work to do

Background (2)Literature Review

Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 34 (1), 199-218Nicol, D & Milligan, C. (2006), Rethinking technology-supported assessment practices in relation to the seven principles of good feedback practice. In C. Bryan & K. Clegg, Innovative assessment in higher education, Routledge.

BackgroundStudent Enhanced Learning through Effective Feedback [SENLEF] project funded by HE AcademyREAP project: www.reap.ac.uk

Rethinking assessment and feedback1. Consider self and peers as much as the teacher as sources of assessment and feedback

Tap into different qualities than teacher can provide Saves time Provides considerable learning benefits (lifelong

learning)

2. Focus on every step of the cycle: Understanding the task criteria (Sadler, 1983)Applying what was learned in action

3. Not just written feedback: Also verbal, computer, vicarious, formal and informal

Seven principles of good feedback

Good feedback:1. Clarifies what good performance is (goals, criteria,

standards).2. Facilitates the development of reflection and self-

assessment in learning 3. Delivers high quality information to students: that

enables them to self-correct4. Encourages student-teacher and peer dialogue

around learning5. Encourages positive motivational beliefs & self

esteem6. Provides opportunities to act on feedback7. Provides information to teachers that can be used

to help shape their teaching (making learning visible)

Source: Nicol and Macfarlane-Dick (2006)

Good feedback:1 Clarifies what good performance is (goals, criteria,

standards).

Examples

Students derive criteria from exemplars (e.g. Essays)Students create problems (e.g. MCQs)for others to solve (Engineering)

Good feedback:2. Facilitates the development of reflection and self-

assessment in learning

• Students provide an abstract with an essay assignment

• Provide written explanation of the concepts underpinning a set of problem-solving questions

OR • Identify what is strong and weak against criteria

when they hand in an assignment (report, essay)• Evaluate the elegance of different solution

pathways to a problem

Good feedback:3. Delvers high quality feedback information to

students: that enables them to self-correct

• Students request feedback when hand in assignment

• Teacher provides ‘feed forward’ rather than feedback

• Focus feedback on skills and on students’ self-assessment abilities

OR• Don’t give feedback – point to resources where

answer/issue can be elaborated

Good feedback:4. Encourages teacher-student and peer dialogue

around learning

T-S dialogueDiscussions of feedback in tutorialsFeedback intensivesPeer dialogueCollaborative assignments (discussed later)Electronic voting methods – polling and peer discussionStudents reviewing each others’ work

Good feedback:5. Encourages positive motivational beliefs and self-

esteem

Focus students on learning rather than on marksEmphasise mistakes are part of learningAlign formative and summative tasksUse authentic and group tasks

Reader responsive feedback (non-evaluative)

Good feedback:6. Provides opportunities to act on the feedback

Provide feedback as action pointsDrafts and redrafts with feedback (new assignment)Reward use of feedback in a different task

Good feedbacks:7. Provides information to teachers that helps them

shape their teaching

Requested feedback Just-in-time teaching – using online testsElectronic voting methods allow dynamic adaptationOne-minute papersDiscussion boards

Application of principles

Problems or bottlenecks Remedies (drawn from the principles)

* Learners don’t understand the assessment criteria so they under-perform* Difficult to provide varied and rich feedback* Learner’s perceive little opportunity to act on feedback* Learners appear too dependent on their teachers* Learners are doing little work most of the time* Teachers don’t get enough information to adapt teaching to learners’ needs

•Active engagement with criteria

•Peer dialogue and feedback

•Inter-relate assignments or drafts and redrafts•Enhance reflection and self assessment•Lots of assignments evenly spread through through the year•Online tests and short one-minute papers

Two meta principles

Meta-PRINCIPLE 1: time and effort on task (structured engagement) i.e. steers on how much work to do and when – Gibbs and Simpson 4 conditions

Meta-PRINCIPLE 2: developing learner self-regulation (empowerment/self-regulation) i.e steers to encourage ownership of learning – the seven principles discussed above.

Case examples from REAP – applying these conditions/ principles

Questions and Discussion

Discussion Points

1. What do you think about these principles?2. Can you share some interesting examples

of your applications of them?3. Are there any gaps – principles you might

wish to include?

Example 1:Psychology

Psychology

560 first year students6 topic areas (e.g. personality, classical conditioning), 48 lectures, 4 tutorials, 12 practicalsAssessment; 2 x MCQs (25%), tutorial attendance (4%), taking part in experiment (5%), essay exam (66%)

Problems identified

No practice in writing skills but required in the examMore detail provided in lectures than mentioned in exams (not enough independent reading)No feedback except on MCQs (percent correct)Didn’t want to increase staff workloadWanted to improve overall exam marksAnd standard of entrant to second year

Discussion Point

What would you do to improve the student experience in first year psychology?

You can use any technology or method (or combination of methods) but you must consider costs and staff time constraints

What principles might apply?

Psychology Redesign

Discussion board in WebCTStudents in 85 discussion groups of 7-8, same groups throughout yearAlso open discussion board for classFriday lectures dropped Students discover for themselves through collaboration what would have been presented in the Friday lectureSeries of online tasks

Structure of group tasks

6 cycles of 3 weeks (one cycle x major course topic)

• First week: ‘light’ written task (e.g. define terms) = 7 short answers (all answer)

• Second week = guided reading • Week three: ‘heavy’ written task: students

answer guided questions and then collaborate in writing a 700-800 word essay.

Within each week:• The Monday lecture – introducing material• Immediately after lecture, task posted online –

for delivery the following Monday• Model answers (selected from students) posted

for previous week’s task

The teaching role

Participation in the discussions was compulsory but not marked (in subsequent years there is 2% mark for participation)The course leader provided general feedback to the whole class – often motivational He encouraged students to give each other feedbackThe group discussions were not moderatedAround 8 teaching assistants monitored the discussions and reported non-participation to the teacher

Guidelines

Comment constructively on each other’s work. If you think that someone has missed a detail or is confused then HELP - point out how an answer might be better. This isn’t showing off – it’s being supportive – don’t leave someone stuck with half an answer or one that’s wrong or confused just because you don’t want to seem to ‘know better’ – it might be you that needs the help next time. Build answers in your online group discussion space, i.e. ‘show your working’ online where possible so that you can keep track of your progress as a group (and also so that Teaching Assistants can check on who is doing what and who isn’t).

Source: Jim Baxter, Psychology, Strathclyde University

 

 

Online Project 1 – Classical Conditioning Phenomena.

Each Group Member should read the Passer chapter. Satisfy yourself that YOU can answer ALL of the questions below. Then agree as a group who will post the final answer. Build answers in your online group discussion space, i.e. ‘show your working’ online where possible  

Project 1 is to answer these questions as fully as you can: 

1) What type of response is susceptible to Classical Conditioning? 2) Why does Extinction occur? 3) What is Spontaneous Recovery? 4) What does the phenomenon of Spontaneous Recovery tell us about the nature of Extinction in Classical Conditioning? 5) etc. 

An example of ‘heavy’ task

The Task – 800 word essay:

Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of personality. Are the theories incompatible?readings suggestedquestions provided – all should tryand advice on how to divide task given

Relation to the Gibbs & Simpson’s four assessment conditions

1. Tasks require significant study out of class (condition 1)

2. Tasks are distributed across topics and weeks (condition 2)

3. They move students progressively to deeper levels of understanding (condition 3)

4. There are explicit goals and progressive increase in challenge (condition 4)

Relation to 7 feedback principles

1. Standard format and model answers provide progressive clarification of expectations (principle 1)

2. Students encouraged to self-assess against model answer (principle 2)

3. Course leader provides motivational and meta-level feedback and selects model answers (principle 3)

4. Online peer discussion aimed at reaching consensus is core feature of design about response (principle 4)

5. Focus on learning not just marks, sense of control/challenge enhanced motivation (principle5)

6. Repeated cycle of topics and tasks provide opportunities to act on feedback (principle 6)

7. VLE captures all interactions allowing course leader to monitor progress and adapt teaching (principle 7)

Benefits

Students worked exceptionally hardWritten responses of exceedingly high standard Discussions about learning and leaner responsibilityHigh levels of motivation: atmosphere in class improvedOnline interactions showed powerful ‘scaffolding’ and community buildingFeedback possible with 560 students: peer and self-feedback (model answers)Easy for tutors to monitor participationImproved mean exam performance (up from 51-57%, p<0.01) weaker students benefit most

Has it worked?

I read more about Psychology and read it earlier in each semester than I would have done without

the online projects

0

10

20

30

40

50

60

1 2 3 4 5

agreed-disagreed

Nu

mb

ers

I learned more in Psychology because of online projects than I did in my other subjects

0

10

20

30

40

50

60

1 2 3 4 5

agreed-disagreed

nu

mb

ers

Series1

I found that reading other people's contributions helped me understand Psychology

010203040

50607080

1 2 3 4 5

agreed-disagreed

nu

mb

ers

Series1

The feedback based on other students' work helped me understand how to improve my own

answers

0

10

20

30

40

50

60

1 2 3 4 5

agreed-disagreed

nu

mb

ers

Series1

What did we learn from the REAP case studies?

Use of simple technologies (discussion board)Considerable thought gone into the learning design [which is transferable]The drivers were learning improvements rather than technology (context of use)Key finding across studies was need to balance structure and learner controlAn important finding was the way that the social and the academic processes were shown to be mutually supportive

Guidelines for Implementation

1. A single principle or many?2. Tight-loose – maintain fidelity to the principles

(tight) but encourage disciplines develop their own techniques of implementation (loose)

3. The more actively engaged students are, the better the design

4. Balance teacher feedback with peer and self-generated feedback

5. Focus on developing students’ own ability for critical evaluation

6. Create opportunities for ‘learning communities’ to develop

7. Share and get feedback on your learning designs

Developments since REAP

Principles of Assessment and Feedback approved by Senate (2008)Use of principles to inform curriculum renewal and QAA processes‘Feedback as Dialogue’ campaign with students and staffPiP project: Building a system to support the re-design of module/programme approval and review processes (www.principlesinpatterns.ac.uk )PEER Project (Peer Evaluation in Education Review)

PEER Project: Core Ideas

The development of self-regulation in higher education requires that:

1. Students must learn to critically evaluate the quality and impact of their own work both during and after its production (e.g. academic texts, problem solutions, designs)

2. Enabling condition for 1 – that there are many opportunities for students to critically evaluate their own work (self-review) and the work of others (e.g. peer review)

Ref: Sadler (2010) Beyond Feedback: Developing students’ abilities in complex appraisal

Some benefits of Peer Review

1. Generating feedback more powerful than receiving it 2. It is cognitively more demanding: cannot be passive3. Puts student in the role of teacher4. Students actively exercise criteria from many

perspectives5. See a wide sample of work produced by other

students 6. They learn that in complex tasks quality can be

produced in different ways 7. Receive a greater variety of feedback thus indicating

how different reviewers perceive their work.

See Sadler, R (2010) and Nicol, D. (2010) and www.reap.ac.uk/PEER.aspx

Informal feedback Formal feedback (explicit)

Teachers give feedback

Peers give feedback on other students work

Students evaluate quality or impact ofown work

Feedback procedures should help students form accurate perceptions of their abilities and establish internal standards with which to evaluate their own work (after Mentkowski and Associates (2000 p82)

Sources

Learning Outcome:Students learn to critically evaluate own and others’ work

Feedback Development Matrix (after Yorke, 2009)

e.g. teachers answer students questions in class

e.g. students discuss the assignment requirements e.g. students engage in a collaborative task

e.g. students generate an abstract to hand in with an essay

‘Feedback is information that might help students to make improvements in their work’

e.g. teachers write feedback on an assignment

e.g. peers comment on each other’s problem solutions in mathematics

e.g. students evaluate the strengths/weaknesses of their essay against criteria.

Some of my PublicationsNicol, D (2010) From monologue to dialogue: Improving written feedback in mass higher education. Assessment and Evaluation in Higher Education. 35(5), 501 -517 Nicol, D and Draper, S (2010), A blueprint for transformational organisational change in HE: REAP as a case study (see reap.ac.uk website)Nicol, D (2009), Transforming assessment and feedback: Enhancing integration and empowerment in the first year, Published by Quality Assurance Agency, Scotland(http://www.enhancementthemes.ac.uk/documents/firstyear/FirstYear_TransformingAssess.pdfNicol, D (2009), Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3), 335-352Nicol, D (2007), Laying the foundation for lifelong learning: cases studies of technology supported assessment processes in large first year classes, British Journal of Educational Technology, 38(4), 668-678Nicol, D (2007) E-assessment by design: using multiple-choice tests to good effect, Journal of Further and Higher Education.31(1), 53-64.Nicol, D. & Milligan, C. (2006), Rethinking technology-supported assessment in relation to the seven principles of good feedback practice. In C. Bryan and K. Clegg, Innovations in Assessment, Routledge.Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), 199-218.See also www.reap.ac.uk for copies.