Slide 1

EVS: The Good, the Bad and the UglyMichaela Cottee & Fiona RobinsContextEVS = Electronic Voting System(s) Used with TurningPoint software, which links with Powerpoint presentations.

University-wide, centrally funded, assessment project during 2010/11.UHBS given 1000 EVS handsets for use in Semester B.500+ EVS handsets were issued to Level 4 students.Also used with Level 5 students.Additional handsets were used for one-off test and revision sessions (various levels).EVS: Four advantagesPromotion of active learning [through anonymity]Generation of [prompt] student feedbackGeneration of staff feedbackEncourages attendance and engagement

Iain Cross, Life SciencesBlended Learning in Practice May 2011

EVS: Good practiceSeven principles for good practice in UG education:EVS supports active learning through anonymityEVS facilitates prompt student feedbackEVS encourages contact between students and staffEVS develops reciprocity and cooperation among studentsEVS emphasises time on taskEVS communicates high expectationsEVS respects diverse talents and ways of learningChickering & Gamson (1987)

OverviewFiona Robins4BUS0154 Decision Making Tools for BusinessN = 82 studentsEVS used in LECTURES to provide both formative and summative feedbackMichaela Cottee4BUS1017 Quantitative Analysis for Accounting and FinanceN = 146 studentsEVS used in TUTORIALS to monitor attendance and to provide summative feedbackFiona RobinsUsing EVS in LecturesPresentationUsing EVS in LecturesEVS used in a level 4 quantitative moduleEVS accounted for 5% of the marks for the moduleUsed for both formative and summative feedback

Why did I use EVSTo improve attendanceNumber of Lectures missedFinal Grade (%)Why did I use EVSTo check understanding

Basil ... there is too much butter on those traysManuel ...no no, senor, not on those tres .... no sir un, dos, tres

Why did I use EVSTo inject some fun

The goodFeedback to the lecturer on students learningInstant feedback to student on their understandingCorrect their understandingCement learning three times to go into brainRecords resultsThe Bad

Time taken from lectureSome students too embarrassed to vote

Stressful to manage lecture and EVSThe Ugly

Managing the TechnologyThe Ugly Number of Lectures missedFinal Grade (%)Attendance didnt improve

Michaela CotteeUsing EVS in TutorialsAssessment diet20% EVS tests (8 weekly in-class tests best 4 results)20% StudyNet quizzes (9 weekly on-line quizzes)20% Mid-term test (written)40% End-of-module test (on-line multiple-choice)Module Handbook infoThere will be 8 weekly in-class tests requiring the use of the EVS handsets. The best 4 results will be used to determine the mark for this assessment. The tests will be in the form of multiple choice questions which must be answered using the EVS handset. Please note:

You should bring your EVS handset (and calculator) with you to every tutorial.You must attempt a test in order to be recorded as present at that tutorial. (In accordance with Business School policy, students who miss more than three tutorials may fail the module through non-attendance.)No allowance will be made for forgetting your EVS handset or if the battery runs out.Lending your EVS handset to another student will be classed as cheating.As the best 4 results out of 8 will be used, if you miss one week (e.g. through illness) we will take the best 4 results out of 7.You will be loaned an EVS handset for the duration of the module. Failure to return it at the end of the Semester will result in a debt to the University.

Solve the following equation for x:x 4 = 22244None of the above

45If a Normal distribution has mean 10 and standard deviation 2, what is P(X < 12)?0.15870.46020.53980.8413None of the above

60Test InstructionsYou will need a pen, paper and calculator.You will need a copy of the standard Normal distribution table.You will need to use the following formula write it down now:

Performance (excluding zeros)End-of-module test average: 54%Mid-term test average: 59%StudyNet quiz average: 69%EVS test average: 79%

Overall average: 62%

Pass rate (before resits): 88%

Tutorial Attendance* Test 8 took place on the last Friday afternoon of term!Test NumberNumber AbsentPercentage Attendance13079%22980%33377%41888%53576%62086%74867%87048%*Attendance & EVS Test PerformanceN = 146Number of absencesNumber of students Best 4 average03589%12787%23781%32477%41464%5349%6426%70-820%The Good...The system lends itself to multiple choice testing as does Quants.Quick and easy to create and incorporate questions into Powerpoint using TurningPoint.The students had very few problems learning to use the EVS handsets.Taking the best X out of Y results gave me a very easy way of coping with missed tutorials / forgotten handsets / faulty handsets / etc.Improved attendance at tutorials and easier monitoring of attendance.Good student engagement.

The Good...Students had EVS tests on all but two topics and so had to revise as they went along.Students met each topic up to FIVE times first of all in the lecture, then in the tutorial, then in the StudyNet Quiz, then in the EVS test and then finally in either the mid-term test or the end-of-module test.Good marks both for the EVS tests and overall.Weekly feedback via StudyNet League table (using SRNs).The Bad...Getting started one more thing to learn incorporate into a new module in a very busy Semester!I didnt seek feedback from students about the use of EVS in the module.I didnt use the results to inform my own teaching or to chase or support weak students.The testing format suited some types / styles of question better than others.NB. The above points reflect badly on me, but not on the use of EVS!

The Ugly...It took several weeks for all students to collect handsets from SSG. The tutors had to be trained in the use of EVS including new tutors who took over towards the end of the module.The tutors felt that the EVS tests ate into the tutorial time and distracted from the tutorial activity.There was signal interference between the tutorial rooms - but coping with the consequences of this was thought to be easier than trying to get the students to change channel on the EVS handsets!

The Ugly (continued)...Reports from TurningPoint were only partially helpful (but not necessarily TurningPoints fault!)Results had to be entered manually into a spreadsheet.Good support available within UHBS (Glenn Alie, Karen Robins, Richard Balawender, etc.) but not always when it was needed! Nothing went seriously wrong the responses were successfully saved each week. Phew!Allowance could not be made for students who forgot their handsets or whose handset wasnt working. Inclusive practice: How to cope with extra time students?

A Third PerspectiveKaren Robins5BUS0133 OR ModelsN = 64 studentsEVS used in lectures to provide summative feedback best 5 out of 7 (worth 10%)

Student Feedback Qualitative & QuantitativeN= 44 respondentsStudent FeedbackEVS is easy to use 98% agreeI would recommend using EVS for this module next year 95% agreeI would recommend use of EVS technology in other Business School modules 90% agreeUsing EVS helped my understanding of the module 74% agreeI was happy to be provided with an EVS handset, knowing that if I did not return it I would have to pay 35 to replace it 40% agree; 60% disagreeWeekly EVS questions were the only reason for attending lectures 28% agree; 72% disagreeTo use EVS technology the user must be computer literate 21% agree; 79% disagree

Negative CommentsThe software would not always workIts annoying because lectures are not meant to be compulsory but I have to attendIf lost the handset it is too expensive to pay for it [35]Sometimes gets confusing to operate (changing channels)Usually forget to take it to the classWhen there were problems with the pad it either took too long to get fixed or trouble to get help and a new oneIt keeps me attending lectures and puts me under pressure when I am late, plus its just one lecture a week which is not friendly!!!!Positive CommentsSimple and straightforward to use the handsetsA good way to enhance understanding by ensuring revision on a weekly basisIs fun at the same time as it is when working hardHelps the teacher to have a better understanding of which students struggle more than othersUsing EVS makes individuals come to the lecture as it counts towards the final gradeIts something new and innovativeSave paper Save the environmentQuick answers/results