Outline

•Introduction and Definitions of CAA•Assessment types•Question types CAA can address•Tips for writing Q’s•Benefits of CAA•Limitations of CAA•Problems with integration of CAA•Cases Studies •CAA examples•CAA and Disability•References•Discussion

‘Assessment consists of taking samples of behavior at a given point in time and estimating the worth of those behaviors. Assessment is the engine that drives a great deal of students learning. Therefore the underlying assumption of assessment is that it provides a representative sample of behavior of the person being assessed’’ (Mogey, N and Watt H. 1996).

‘Computer Aided Assessment refers to any instance in which some aspect of computer technology is deployed as part of the assessment process’’ (Atkinson and Davies, 2000)

Diagnostic Assessment: It is used to identify the strengths and weaknesses of the students.

Formative Assessment: this occurs throughout a course and provides feedback to students to help them to improve on their performance. It is made up of three areas,1) Diagnosing student difficulties 2) Measuring improvement over time3) Providing information to inform students how to improve their learning.

Summative Assessment: generally occurs at the end of a systematic and incremental series of learning activities or learning modules. It formally assesses the student’s knowledge and understanding- Scaffolding – combining Summative and Formative Assessment

Objective tests require a user to choose or provide a response to a question whose correct answer is predetermined (McKenna and Bull 2004).

•Multiple Choice•Fill in the Blanks•True / False•Multiple Answer•Inputting text (words, numbers or formulae)•Free format answers / essays•Labeling questions•Matching exercises•Ranking Questions•Field Simulations•Identify or manipulate images, plot points on graphs.

STEMS RESPONSES KEY AVOID

Be clear and concise

Should be equally likely

In a varied position

Double negatives

Not contain ambiguities

Approx the same length

Correct answer

‘None or all of the above’

Not be negative if possible

Internally consistent

Clues

Stem – introductory Q or incomplete statementResponse – suggested answers or completion of stemKey – correct answerDistracter – incorrect answers

•Lecturers can monitor student’s progress through more frequent assessments.

•Students acquire It Skills

•Detailed specific and timely feedback is available instantly.

•Automated marking of scripts which makes it quicker for both staff and students reducing workload.

•Potential to include graphics and multimedia allows for the inclusion of questions which could not be asked by paper based exams.

•Aids the transmission and administration of marks which can be automatically entered into a database.

•Eliminates the need for double marking and marking is not prone to human error.•Randomisation of questions which reduces the potential of cheating

•Initial implementation of a CAA system can be costly and time consuming

•Hardware and Software must be carefully monitored to avoid failure during examinations

•Students must acquire adequate IT skills and experience of the assessment type in advance of the exam.

•Assessors and invigilators need training in assessment design, IT Skills and examinations management.

•Question types supported by CAA are limited.

•Misleading Clues / Disparity

•Plagiarism / Security

•Interoperability

Brown, S, Bull, J and Race, P 1999) have discussed some of the most common perceived problems with integrating technology into both courses and assessment:

How can I ensure

security?

None of the rest of my course is Computer Based….

I’ve only got 20

PC’s in the lab and

I have 40

students??

Things are working fine at

the moment why should I

change…..

I don’t have time to set it up!!

Great writers ‘fail’ online test

Science GCSE with online exams:

Online marking of exams ‘faulty’:

‘Online marking of exams is faulty’

‘Great writers ‘fail’ online test’’

‘’Less is more for online marking’’

The number of students registering with a disability to higher education is increasing.Majority of students register dyslexia or blind/partially blind as disability.Where does CAA come into this?

•When designing for CAA or online courses it is now essential that text equivalents are given for images and video, transcripts of sound recordings and ensuring information conveyed by colour is available to those with colour deficient sight. •It can benefit students when it comes to exams as a student unable to sit a long exam paper or write an exam paper may be able to operate a switch or keyboard to indicate correct answers.•It is not enough to simply redesign a summative assessment to meet the needs of a disabled student. If other students have been receiving formative assessment it can be seen that the disabled student is at a disadvantage without any practice.

Lawrie Phipps and Dave McCarthy – TechDis Study:

Hot Potatoes Software

Question Mark Perception

Respondus – Question bank network

Web – CT

CALM – Computer aided Learning in Maths part of the PASSIT initiative in Scotland.

Thank you for Listening.

•Boud, D. and associates (2010) Assessment 2020: Seven propositions for assessment reform in higher education, Sydney: Australian Learning and Teaching Council•Boud, D (1995b), 'Assessment and learning: contradictory or complementary?' in P.Knight (Ed.) Assessment for Learning in Higher Education. London: Kogan Page. pp 35-48.•Brown, S, Bull, J and Race, P (1999). Computer Assisted Assessment. London: Kogan Page. p8-17•Bull J, McKenna C (2004). Blueprint for Computer Assisted Assessment. London: RoutledgeFalmer. P2-18.•Dunn, L, Morgan, C, O'Reilly, M, Perry, S (2004). The student assessment handbook. London: RoutledgeFalmer. p18-22. P41 -46•Maier, P, Warren, A (2000). Integrating Technology in learning and teaching. London: Kogan Page. p139 -141.•Mogey, N and Watt H. (1996). The use of computers in the assessment of student learning. In: Greg Stoner IMPLEMENTING LEARNING TECHNOLOGY. Edinburgh: Institute for Computer Based Learning Heriot-Watt University. 50 - 57.•Rowntree, D (1987) Assessing Students: How shall we know them? Revised Ed. Kogan Page, London, ISBN 0-89397-271-1.