PHEE April 2001

Meeting the Assessment Challenge:

finding practical approaches

Su White and Hugh Davis

IAM Learning Technologies:

ECS, The University of Southampton

PHEE April 2001

e3an Electrical and Electronic

Engineering Assessment Network

a three year HEFCE project fromthe Fund for the Development of Teaching and Learning (FDTL)

PHEE April 2001

e3an - the participants

South Coast HEIs Southampton Portsmouth Bournemouth Southampton Institute consultants across the sector plus steering group (IEE, CAA Centre)

PHEE April 2001

the importance of assessment - 1

QAA feedback says we are all bad at feedback

QAA wish to see how assessment links to learning outcomes

subject benchmarks National Qualifications Framework

PHEE April 2001

the importance of assessment - 2

assessment shapes students’ learning the importance of dialogue and feedback early diagnosis of problems

lecturerstudent

PHEE April 2001

e3an – the context …

much existing good practice expertise built up over years solutions for particular contexts technology and people approaches identify, explore, acknowledge,

disseminate

work collaboratively across sector

PHEE April 2001

e3an - the plan

question banks as a focus peer reviewed test banks of questions for

electrical and electronic engineering questions produced and reviewed by

paid consultants

supporting materials on methods and practices, workshops and consultancies

PHEE April 2001

e3an - the plan...

questions banks may be:multiple choice objective teststext based traditional testsself assessment, feedback, course

assignments, exam questionsmainly for formative and diagnostic

assessment – but also as source of ideas for summative assessment

PHEE April 2001

so what is innovative about this?

it has never been done like this before the questions will form a yardstick against

which achievement may be gauged the project aims to build a network of

“champions” across the sector the project focuses on assessment

outcomes rather than technology

PHEE April 2001

themes and teams

theme leader plus expert consultants categorising syllabus cash available for travel plus small fee local teams in phase one - trialling the

process, four themes national teams in phase two, eight

themes

PHEE April 2001

theme process

Working together the theme teams: identify question targets discuss levels and approach agree division of tasks write and review questions trial and evaluate questions

PHEE April 2001

phase one themes

Analogue Electronics Digital and Micro Processors Circuit Theory Signal Processing

PHEE April 2001

phase two target themes

Power Electronics

Electromagnetism

Computer

Programming

Control Engineering

Maths

Telecomms

Physics/

Semiconductors

Datacomms

NB all open for discussion

PHEE April 2001

database and interoperability

XML database IMS QTI interoperability standards ++ objective tests for common test engines collaboration with other projects print as the lowest common denominator some examples...

PHEE April 2001

project questions

interoperability standards - reality of vapourware?

how should authors produce questions for maximum interoperability?

are some question types more interoperable than others?

how do we choose a system for interoperability?

PHEE April 2001

more project questions

creating test banks in a cost-effective manner

choosing themes (usefulness, greatest gains, ++)

classifying questions - level of skills , difficulty, cognitive skills, time required

subdividing question banks into sub themes

which themes/items suit CAA, which don't how to establish an effective peer review process

PHEE April 2001

looking to the future

greater involvement of existing expertise barriers and drivers for use and

dissemination distribution models continuation growth and maintenance of

the test bank

PHEE April 2001

e3an - the benefits

identifying and spreading good practice evaluated trial use of automated

assessment on campus potential to trial new software (e.g. web

based QM perception) potential time savings students feedback improved (in time for

next QA / academic review

PHEE April 2001

e3an - more details

Su White saw@ecs.soton.ac.uk tel +44 (0)23 8059 4471 fax +44 (0)23 8059 2865 http://www.ecs.soton.ac.uk/E3AN/

please contact us if you would like to be involved :-)