Course Review on Assessment

Professor Brenda Smith

• It is intended to map assessment patterns across a programme

• It looks back at modules over a 3 year degree course to chart a typical students experience

10 steps to auditing assessment across a programme

TESTA: Transforming the experience of students through assessment

• Funded by the National Teaching Fellowship Scheme

• Seeks to develop course assessment interventions and to evaluate them

• Involves 4 universities and led by Graham Gibbs

TESTA Model

It maps:• Volume of summative & formative assessment• Variety of assessment methods• Percentage from coursework and exams• Volume of written and oral feedback• How criteria contribute to student clarity about

goals and standards

Caveats…

• Gives an overview from a course leader or course team perspective

• There is often a gap between documentation & the reality of the student experience

• It works best when triangulated with other data like focus groups with students and the Assessment Experience Questionnaire (AEQ)

TEN STEPS - STEP 1

Select a named programme

What is the typical student cohort• Number of students in each year• What is their previous educational

experience?• What are students career aspirations?

Year 1 Year 2 Year 3

STEP 2

• How many modules are there in the first year of the programme?

• What is their credit weighting?

20 Credit

s40 Credit

s

Business Studies

Year 1

•  Management Accounting•  Contract Law•  Foundations of Marketing•  Economics for Business•  Managing People•  Quantitative Business Methods

STEP 3

Describe the nature of assessment in each module• Type of assessment• Is it formative and summative?

Now tally the summative, formative and variety of assessment methods

STEP 4

• Repeat for the second and third years

• Tally the number of summative, formative and the variety

• Tally the number of assessments by exam, and work out the relative percentage of course work to exams

STEP 5

• For each assessed task estimate how long it takes from the submission of an assessed task to the return of feedback and marks

• Are feedback and marks separated?

• What is the system for the collection of assessed work?

• Is certain feedback and the mark returned faster than others? E.g presentations

STEP 5 continued….

• Are feedback and marks released as provisional?

• Is formative feedback returned more quickly than summative feedback?

• Are there different variables e.g. 80% of written work is returned after 21 days, & 20% is presentation, returned within 1 day?

• WORK OUT THE OVERALL RETURN FACTOR

STEP 6Work out as best you can how much oral feedback a student

is likely to receive each year and over the degree• Tutorial provision before & after assessment points, plus

estimated take up of these % wise & length of tutorial

• You may need to explore how assessment is handed back

• May involve looking at numbers of students called back after a failed submission

• May involve quantifying proportion of oral feedback given through digital technology- (MP3 files & smart phones) & proportion who listen, oral feedback given in whole class sessions and on placements etc

STEP 7

Look at explicit assessment criteria and learning outcomes in the programme documentation

• How do written statements appear in mapping from programme to module to task?

• Is there alignment between tasks & intended outcomes?

• How do students know what ‘good’ is in relation to assessment tasks

STEP 7

continued…

• Does the programme have a system wide approach to developing the student ‘nose’ for what constitutes a good essay, presentation, poster etc?

• How do feedback processes clarify goals and standards?

STEP 8

• Using the sequence guide (given as a separate sheet) map the sequence of different assessment tasks across modules and levels.

• Describe how assessment articulates across the programme , feeding forward into the next assessment, providing practice and allowing students to develop their performance

STEP 9

Gather together 15-20 cover sheets and if possible scripts with comments on them from each year – 1st, 2nd & 3rd.• Count the number of words on the cover sheets and

scripts• Add 1st, 2nd & 3rd year total words• Divide the total by the number of student cover

sheets• Multiply the average number of words of written

feedback by the total volume of summative feedback• This is the total volume of written work a student

might expect over the course of a 3 year programme

STEP 10

• Put all the data together in a document

• Now working with the course team what are you really pleased about?

• What might you want to do differently?

• How might you run some student focus groups to triangulate this information?

Ref: De Tansy Jessop, TESTA

Table 2 Range of characteristics of assessment environments found in different degree

programmes

Characteristic of assessment environment

Minimum Maximum

Percentage of degree marks derived from examinations

17% 100%

Percentage of degree marks derived from coursework 0%

83%

Total number of times work marked per student

11 95

Variety of assessment methods

1 18

Total number of formative-only assessments per student

2 134

Total number of words of written feedback per student

2,700 10,350

Total number of hours of oral feedback per student

3 68

Average number of days between submission of

assignment and feedback

1 28

REF: Graham Gibbs “Using assessment to support student learning” Leeds Metropolitan University

Positive student

learning responsesThe following programme level assessment characteristics

have been found to be associated with a variety of positive student learning responses:

• a high volume of formative-only assessment

• a high volume of oral feedback

• timely feedback

… and to a lesser extent:

• a high volume of written feedback

• a low proportion of marks from coursework

Negative student learning

responses

The following programme level assessment characteristics have been found to be associated with a variety of negative student learning responses:

• a high volume of summative assessment

• a wide variety of types of assessment

• highly explicit criteria and standards

• highly ‘aligned’ assessment (with different assessments and criteria associated with each learning outcome)

Figure 1: A student’s estimate of his weekly study effort

(in hours) on a programme with no assignments and an exam

in week 12. Total study effort = 68 hours

Figure 2: A student’s estimate of her weekly study effort (in hours) on a programme with three

assignments due in weeks 4, 7 and 10 and an exam in week 12. Total study effort = 78 hours

Ref: Using assessment to support student learning (2010)By Graham Gibbs ISBN 978-1-907240-06-5

TESTA Programme Portrait

Wombledown University, Creative Arts

Degree (CAD)

TESTA Programme Portrait

Wombledown University, Creative Arts

Degree (CAD)

• On average, a CAD student received 214 words of

feedback on each summative assessment, low

oral feedback from tutors, and no formative

assessment was formalised in the

documentation. The programme was high in

variety, and it was the quickest on return of

assessed work

Focus Group Data

USE OF FEEDBACK• Students use feedback to feed-forward• For some module feedback does not feed forward• Peer feedback is highly valued & integral to the

program• It takes time and practice to give good feedback• Low stakes, informal feedback is valued• Comments without marks increase use of feedback• Marks without feedback diminish use of feedback