Learning Outcomes: Learning Outcomes: Keeping it RealKeeping it Real

Liz WarrLiz Warr

Eduspeak Eduspeak

Dialect particularly spoken in Educational Dialect particularly spoken in Educational Development Units, including favoured Development Units, including favoured phrases such as intended learning phrases such as intended learning outcomes, assessment criteria, outcomes, assessment criteria, constructive alignment, cognitive constructive alignment, cognitive dissonance and performance indicatorsdissonance and performance indicators

Aim of the SessionAim of the Session

To familiarise participants with the To familiarise participants with the pedagogic principles underpinning the pedagogic principles underpinning the construction of Intended Learning construction of Intended Learning Outcomes in the context of their impact on Outcomes in the context of their impact on assessment practiceassessment practice

Learning OutcomesLearning Outcomes

At the end of the session participants will be able At the end of the session participants will be able to:to:Apply the principle of ‘constructive alignment’ to Apply the principle of ‘constructive alignment’ to module designmodule designDevise module specifications that meet the Devise module specifications that meet the needs of key stakeholdersneeds of key stakeholdersSelect assessment criteria that measure Select assessment criteria that measure achievement of learning outcomesachievement of learning outcomesWrite learning outcomes that reflect the Write learning outcomes that reflect the academic level of student learningacademic level of student learning

Assessment CriteriaAssessment Criteria

Survive Scrutiny of Validation and/or Survive Scrutiny of Validation and/or Accreditation Panel Accreditation Panel

Gain Approval from ASCGain Approval from ASC Receive positive Feedback from QAA Receive positive Feedback from QAA

AuditorsAuditors

Learning OutcomesLearning Outcomes

Statements of what a learner is expected Statements of what a learner is expected to know, understand or be able to to know, understand or be able to demonstrate after completion of a process demonstrate after completion of a process of learningof learning

Represent a major achievementRepresent a major achievement

(a significant capability, process or task)(a significant capability, process or task)

AssessmentAssessment

Assessment is a generic term for a set of processes that measure the outcomes of students’ learning in terms of knowledge acquired, understanding developed and skills gained.

(QAA, 2000)

Assessment CriteriaAssessment Criteria

Descriptions of what the learner is Descriptions of what the learner is expected to do, in order to demonstrate expected to do, in order to demonstrate that a learning outcome has been that a learning outcome has been achievedachieved

Set at a threshold level of achievement: Set at a threshold level of achievement: performance in excess of this can be performance in excess of this can be further differentiated by grading criteriafurther differentiated by grading criteria

Credit LevelCredit Level An indicator of relative demand, An indicator of relative demand,

complexity and depth of learning, and of complexity and depth of learning, and of learner autonomylearner autonomy

Generic level descriptors are available for Generic level descriptors are available for the full spectrum of Further and Higher the full spectrum of Further and Higher Education. Education.

(Credit levels are not intrinsically linked to (Credit levels are not intrinsically linked to years of full-time study)years of full-time study)

Assessment DesignAssessment Design

Is based on the principles of “Constructive Alignment” where teaching methods and assessment processes are aligned with the learning outcomes so that all aspects of the teaching system support appropriate student learning

(Biggs, 1999)

Pedagogic arguments for Outcome-Pedagogic arguments for Outcome-based approachbased approach

Promotes active learning Promotes active learning Student-centred rather than tutor focussed Student-centred rather than tutor focussed Provides simple mechanism for setting Provides simple mechanism for setting

and reviewing academic standardsand reviewing academic standards Increases transparencyIncreases transparency Increases flexibility for Accreditation of Increases flexibility for Accreditation of

Prior Learning (APCL and APEL)Prior Learning (APCL and APEL) Facilitates progression and student Facilitates progression and student

mobilitymobility

ModuleModule

A self-contained, formally structured learning A self-contained, formally structured learning experience with a coherent and explicit set experience with a coherent and explicit set of learning outcomes and assessment of learning outcomes and assessment criteria which determine its credit level. It is criteria which determine its credit level. It is the smallest entity for which credit may be the smallest entity for which credit may be awarded and will be assigned a volume of awarded and will be assigned a volume of credit which defines the nominal size of the credit which defines the nominal size of the module (estimated notional learning hours for module (estimated notional learning hours for its completion)its completion)

Notional Learning TimeNotional Learning Time

The number of hours which it is expected The number of hours which it is expected a learner will spend, on average, to a learner will spend, on average, to achieve the specified learning outcomesachieve the specified learning outcomes

The most widely used credit : learning The most widely used credit : learning ratio is 1 credit :10 hours. ratio is 1 credit :10 hours.

1 full time undergraduate academic year = 1 full time undergraduate academic year = 120 HE credits or 1200 notional learning 120 HE credits or 1200 notional learning hourshours

1 Calendar Year = 180 HE credits or 1800 1 Calendar Year = 180 HE credits or 1800 notional learning hoursnotional learning hours

Module DevelopmentModule Development

Aims of Module

Make Assessment Decisions Design Assessment Methods and Tasks

Define Assessment Criteria/ Grading CriteriaDevelop Learning and Teaching Strategies

Provide Feedback for Students

Evaluate and Revise

Define Learning Outcomes

Bloom’s Taxonomy of Learning ObjectivesBloom’s Taxonomy of Learning Objectives

Evaluation

Synthesis

Analysis

Application

Comprehension

Knowledge (Acquisition)

(Bloom, 1956 cited in Brown, 2001)

Hierarchy of External InfluencesHierarchy of External Influences Sector: - Framework for Higher Education Sector: - Framework for Higher Education

Qualifications (FHEQ) Qualifications (FHEQ)

- Qualification descriptors- Qualification descriptors

Discipline: - Subject Benchmark Statements (QAA)Discipline: - Subject Benchmark Statements (QAA)

Programme: - Programme Specification (Institution)Programme: - Programme Specification (Institution)

Module: - Generic level descriptors (HE Credit Module: - Generic level descriptors (HE Credit Framework for England) Framework for England)

- Grading Criteria (Institution)- Grading Criteria (Institution)

Threshold vs Higher Level AchievementThreshold vs Higher Level Achievement

AssessmentAssessment Criteria test ILO’s at threshold levelCriteria test ILO’s at threshold level If achieved - Module PassIf achieved - Module Pass

- Award of Academic Credit- Award of Academic Credit

GradingGrading criteria measure achievement above criteria measure achievement above threshold (eg Pass, Merit, Distinction, 3threshold (eg Pass, Merit, Distinction, 3 rdrd Class, Class, 22ndnd Class, 1 Class, 1StSt Class Class

Higher level achievementHigher level achievement does notdoes not result in result in award ofaward of additional Credit additional Credit

Building in Higher Level SkillsBuilding in Higher Level Skills

Either Either Specify in the Learning Outcomes (must be assessed, Specify in the Learning Outcomes (must be assessed,

must be passed)must be passed)Or Or Specify in the rationale and context section and build into Specify in the rationale and context section and build into

grading criteria (should be assessed, may not need to be grading criteria (should be assessed, may not need to be passed)passed)

Often used to justify the use of 40% as threshold passOften used to justify the use of 40% as threshold pass(remaining 60% can reflect achievement of higher level (remaining 60% can reflect achievement of higher level

skills / graduate attributes set out in generic institutional skills / graduate attributes set out in generic institutional assessment criteriaassessment criteria

Assessment PracticeAssessment Practice

Assessment should be an integral part of the curriculum delivery on an ongoing basis and must be timed carefully so that students obtain maximum benefit and so that both students and staff have manageable workloads

It is good practice to map out the assessment for a whole programme:

to ensure comprehensive coverage of the Programme Learning Outcomes

to ensure students experience a balanced diet of assessment methods

to ensure realistic time-scales are set; for completion by students, marking by assessors and provision of feedback to the learners

Assesperanto Assesperanto

the dialect which needs to be learned by the dialect which needs to be learned by students if they are to succeed in students if they are to succeed in assessments - particularly those requiring assessments - particularly those requiring advanced competences with academic advanced competences with academic EnglishEnglish

FeedbackFeedback

Is like fish – only good when its fresh

Assessment PracticeAssessment Practice

“ “Institutions should ensure that the amount Institutions should ensure that the amount and timing of assessment enables and timing of assessment enables effective and appropriate measurement of effective and appropriate measurement of students’ achievement of intended students’ achievement of intended learning outcomes”learning outcomes”

(QAA Code of Practice on Assessment)(QAA Code of Practice on Assessment)

When Writing Learning OutcomesWhen Writing Learning Outcomes

Avoid words like:Avoid words like: know, understandknow, understand be familiar with be familiar with become acquainted withbecome acquainted with have a good grasp ofhave a good grasp of obtain a working knowledge ofobtain a working knowledge of be aware ofbe aware of believebelieve be interested inbe interested in

When Writing Learning OutcomesWhen Writing Learning Outcomes

Consider using words like:Consider using words like: DescribeDescribe EvaluateEvaluate ExplainExplain IdentifyIdentify DistinguishDistinguish DesignDesign constructconstruct solvesolve

ReferencesReferencesBiggs, J (1999) Teaching for Quality Learning at University. Buckingham UK: Open University Press

Brown, G (2001) Assessment: A Guide for Lecturers. York: LTSN Generic Centre Assessment Series Guides 2001

Brown, S and Glasner, A (2003) Assessment Matters in Higher Education, 3rd Edition. Buckingham UK: Open University Press

Gosling, D and Moon, J. How to use Learning Outcomes and Assessment Criteria. London:SEEC

QAA (2006) Code of Practice for the assurance of academic quality and standards in Higher Education. Section 6: Assessment of students QAA (2008) The Framework for higher education qualifications in England, Wales & Northern Ireland

www.qaa.ac.uk/academicinfrastructure/benchmark

QAA (2008) Higher Education credit framework for Englandhttp://phil-race.co.uk/ 

Rust, C., O’Donovan, B & Price, M. (2005) Assessment and Evaluation in Higher Education. Vol 30, No 3 233-241

Facilitese Facilitese

Used by some trainers, often including Used by some trainers, often including phrases such as ‘now let’s go into groups’ phrases such as ‘now let’s go into groups’ and ‘go to the breakout rooms’ and ‘I’ll just and ‘go to the breakout rooms’ and ‘I’ll just unpack that for a minute’unpack that for a minute’

Doing it for RealDoing it for Real

Working in small groups, produce an Working in small groups, produce an outline module specification for an outline module specification for an assigned topicassigned topic

-Aim-Aim

-L.O’s-L.O’s

-Assessment Strategy-Assessment Strategy

-Teaching and Learning Strategy-Teaching and Learning Strategy

-Assessment Criteria-Assessment Criteria

Get RealGet Real

WHAT do you want the Students to be able WHAT do you want the Students to be able to do? to do?

HOW do you want them to demonstrate HOW do you want them to demonstrate that they can do it?that they can do it?

HOW WELL do they have to perform?HOW WELL do they have to perform?

WhatWhat

Swim 50 metersSwim 50 meters

HowHow

In a swimming pool wearing outdoor In a swimming pool wearing outdoor clothesclothes

How WellHow Well

In 5 minutesIn 5 minutes