Online examination: a feasiblealternative during COVID-19

lockdownChoon Yong Alex Tan, Kye Mon Min Swe and Veronica Poulsaeman

Faculty of Medicine and Health Sciences, Universiti Tunku Abdul Rahman –Kampus Bandar Sungai Long, Kajang, Malaysia

AbstractPurpose – The purpose of this case study is to share the authors’ involvement and observation in adoptingonline exam as an evaluation tool, in the hope that others may too be able to adapt and circumvent thislockdown period while bringing a more reliable methodological approach in an online exam.Design/methodology/approach – This is a case study that reports the application of an onlineexamination for a year 3 MBBS Professional Exam during the period of lockdown (movement control). Thiscase highlights the conduct of online theory in the form of single best answer multiple choice questions andonline practical via objective structured clinical examination, without jeopardizing the integrity of the system,and themeasures adapted to mitigate potential academic dishonesty.Findings – The authors’ observation shows that online exam can be conducted in scenario whereconventional exam is not possible.Research limitations/implications – This is an observation in a real-life scenario that discusses thepurpose and the process. A single study may not be sufficient to relate the reliability and generalizability ofthe analysis.Practical implications – The practice of social distancing is a hindrance towards traditional classroomteaching. Online exam has promising potential to be the solution to this problem. The authors havedemonstrated that an MBBS theory and practical exam can be conducted remotely without any physicalcontact, without any compromise in achieving the learning objectives within the curriculum. Medicaleducation can proceed with a new drive while ensuring the learning process continues a stable momentum.Originality/value – Online examination is almost unheard of, especially for undergraduates in the medicalprogramme. This study highlights a plausible approach in the design of the online examination and identifiesthe barriers and favourable edges of this platform.

Keywords COVID-19, Online assessment, E-assessment, Google form, Online OSCE,Online examination

Paper type Case study

IntroductionCOVID-19 infection is an uncontrolled global pandemic (Coronavirus [COVID-19] events asthey happen, 2020). Many countries rely on social distancing to halt the spread of thedisease, including Malaysia. The Movement Control Order was first implemented in March2020, which prohibited nationwide activities on mass gathering, including closure of alleducation centres. Medical institutions are not exempted from this ordeal (Bunyan, 2020).This prompted the Education Ministry to commence online learning till the reopening ofschools (Povera and Yunus, 2020). TheMalaysian Qualification Agency has released a set of

Funding sources: This research received no funding.Conflict of interest: The authors declare no conflict of interest.

QAE29,4

550

Received 9 September 2020Revised 18 January 202121 April 2021Accepted 19May 2021

Quality Assurance in EducationVol. 29 No. 4, 2021pp. 550-557© EmeraldPublishingLimited0968-4883DOI 10.1108/QAE-09-2020-0112

The current issue and full text archive of this journal is available on Emerald Insight at:https://www.emerald.com/insight/0968-4883.htm

guidelines for those in higher education, of which all face-to-face teaching and practicalactivities are to be carried out fully online as well (Malaysian Qualifications Agency, 2020).

The utilization of technology in the education sector is no longer sitting in a luxurioussupportive role but is an indispensable core in the current system. This fusion changes thewhole perspective on education. It opens new opportunities and breaks the barrier oftraditional classroom teaching and learning. This includes the option for paperlessassessments in the likes of knowledge-based assessment (e.g. using multiple choicequestions), performance-based assessment (e.g. using Objective Structured ClinicalExaminations [OSCE] stations) or practice-based assessment (e.g. using portfolios orlogbooks) (Crisp, 2007). Planning for an e-assessment however, as with any assessmentprocess, needs careful consideration in the form of the assessment required, how they relateto the immediate learning objectives/outcomes and the rest of the curriculum and how (andindeedwhether) they are to be completed electronically at all (Ellaway andMasters, 2009).

The implementation of online exams has been reported as early as 2009, which concludedwith favourable results (Schmidt et al., 2009). Recently, Imperial College London justconducted an online exam for 280 final year candidates, reputedly the first ever for medicalstudents globally (Tapper et al., 2020). This dawns an era that marks the application ofonline exams for medical programmes, which presumably requires clinical examination as aprerequisite.

Thus, to safeguard the supply of graduating doctors during this critical period, it ispivotal to carry out a form of plausible evaluation without compromising either the order ofsocial distancing or the standard of the medical programme. Our current technology enablesus to make the transition from the long-established conventional face-to-face teaching toextensively online, if not exclusively, hence enabling the possibility of even conducting anexamwithout the physical presence of the candidates on the campus.

A professional exam for MBBS year 3 candidates was carried out as an onlineexamination (e-assessment) in May 2020. The online examination conducted by theuniversity was in line with the requirements laid out by the Malaysian Medical Council andthe Malaysian Qualification Agency (Malaysian Medical Council, 2020). The assessmentprincipally measured the candidate’s knowledge and skill in accordance to the learningoutcomes. Additional efforts were implemented to mitigate any form of academic dishonestyall through the exam.

ObjectiveThe objective of this case study is to share our involvement and observation in adopting anonline exam as an evaluation tool, in the hope that others may too be able to adapt and tocircumvent this pandemic period while bringing a more reliable methodological approach inan online exam.

Designing systemThe exam was made up of theory and clinical components. The exam coordinator ensuredsufficient invigilators to oversee the candidates and briefed them as to the kinds ofbehaviours and misdemeanours they need to be looking out for. It was also the obligation ofthe exam coordinator to ascertain the availability of technical support to respond quicklyshould there be unforeseen glitch.

In the meantime, the candidates were briefed about the protocol and procedure of theexam, and trial runs were conducted. Feedback was gathered during the trials to improvethe system. As the current lockdown restricted us from recruiting any patients, anyconsenting participants were thus engaged as simulated patients.

Onlineexamination

551

Theory exam or knowledge-based assessmentThe theory exam was conducted online via Google Form. Theory component consists of 200questions in the form of a single best answer (SBA). A total of 13 invigilators were recruitedfor the proctoring of 49 candidates, who were then placed into 13 different groups. Eachinvigilator oversaw a group of four candidates, save the last group which comprised only asingle candidate. Microsoft Teams was the platform of choice to monitor the candidatesconstantly. The whole invigilation process commencing from briefing till end of the examwas recorded and observed by one staff member from the Department of Examinations.

Candidates were required to procure at least two devices for this exam, preferably alaptop and a smartphone. All 49 students fulfilled this prerequisite to sit for the online exam.The purpose of the laptop was to visualize the exam questions. The smart phone was usedas a video camera which would be linked to Microsoft Teams, strategically placed at thecorner back of the candidate; thus, the invigilator would have a view of the laptop screenand the candidate as well. Before initiating the exam, candidates were subjected tomandatory authentication verification with student identification cards and the examinationslip. The rules of the examination were read by the invigilators. In addition to that,candidates were required to show the surrounding area to ensure that the vicinity was freeof any unauthorizedmaterials.

Having completed so, an email would be transmitted to them, which was a link toconfirm their attendance and declaration statement in Google Form. Successful submissionwould initiate another email, which was an encrypted link to their exam questions in GoogleForm. The password was sent by the exam coordinator via WhatsApp to all the invigilatorsand then verbally conveyed to the candidates by the respective invigilators.

The theory exam was made up of Paper 1 and Paper 2. Each paper consisted of 100 SBAquestions, which was fractionated into five parts of 20 SBA questions individually for aduration of 30min. During submission of the answers online, candidates were required tomanually write all their answers in a pre-ready paper, in the event that failure of submissionbecause of technical glitch would still be salvageable. Candidates were required to submitwithin 3min after the exam ended. Any non-compliance would be considered a fail in thatparticular part. Simultaneously, the exam coordinator would be monitoring the responses.Having confirmed that all the responses were received in the system, the exam coordinatorwould subsequently provide a new encrypted link and password for the next part.

Prior to ending the exam, candidates were again asked to reveal the vicinity within theroom and send a snapshot photo of the manually written answers to the invigilator. Duringthis time, candidates were required to enable Microsoft Teams on their laptop instead, as thephone would be used to capture and send the photo (Figure 1).

Clinical exam or performance-based assessmentThe clinical exam was conducted via online in the form of OSCE. Prior to the exam,students’ feedback was taken to make sure about the availability of simulated patients andrelevant examination tools.

There were a total of ten stations conducted in a span of four days. Four circuits would berunning concurrently daily. Each circuit was assigned to an examiner. The evaluation wascarried out with Microsoft Teams as the platform of choice. Before the commencement of theexam, candidates were subjected to authentication verification by displaying their studentidentification card as well as the examination slip. Every candidate within each circuit wasallotted non-identical questions to avoid any potential cheating. The questions wereprojected via share screen in PowerPoint presentation. To enhance the integrity of the exam,the OSCE was recorded and monitored by members from the Department of Exam. Each

QAE29,4

552

candidate was granted 6min duration for non-performing questions and 10min forperforming questions with a simulated patient.

Should a candidate per chance experience any form of disruption of exam, namely,internet disconnection, lagging, blackout and others, the evaluation would be delayed to alater period on the same day using another set of non-identical reserved questions.

At the end of the OSCE exam, the mark sheets were submitted to the exam coordinatorvia email as an encrypted attachment. Throughout these four days, ten OSCE questionswere conducted to a total of 49 candidates involving 22 examiners and four facilitators(Figures 2 and 3).

ResultAnalysis of the exam result yields a positive correlation (p< 0.05) between continuousassessment theory (traditional paper–based assessment) and online theory. There is,however, no significant correlation (p> 0.05) between continuous assessment OSCE(traditional face-to-face OSCE exam) and the online OSCE. Looking into the mean score, theperformance of the students from this present online exam is comparable to the previousbatch who sat for the conventional classroom exam.

Figure 1.Invigilation ofcandidates via

Microsoft Teams

Figure 2.Flow of the online

OSCE exam

Onlineexamination

553

DiscussionThe transition to online teaching and learning is inevitable in the process of modernizationto embrace the future. This global pandemic has served as a catalyst to adopt thisconversion much earlier than expected. Social distancing will carry on, and no matter howefficient this method is in the prevention of disease transmission, it is also a hindrancetowards traditional classroom teaching. Online exams have promising potential to be thesolution to this problem.

The conduct of this online exam has implicated a few areas of technicality to beaddressed. Adequate knowledge on basic computer skill is a prerequisite, which could beeasily rectified with due familiarity eventually. This would mean not just any academic staffis capable of holding the cup of responsibility in preparation of the online exam. Besidesconverting the questions to online format, the trained personnel must also be able to providetechnical support to any candidate who encounters difficulty. Thus, if online exam is to befully used, sufficient tutoring must be provided to lecturers.

The shift to online learning would mean reconfiguring a teaching plan to suit the currentneeds of the syllabus. The available technology enables us to fully use online exam as a formof evaluation in future directions. Other studies have shown effectiveness in the conduct ofonline OSCE in comparison to traditional OSCE (Chaudhary et al., 2017; Kaliyadan et al.,2014). We have demonstrated that OSCE too can be conducted remotely without anyphysical contact or any compromise in achieving the learning objectives within thecurriculum. Medical education can proceed with a new drive while ensuring the learningprocess continues a stable momentum.

It is essential to note the benefits of online exam. Each station has been set up withminimal difficulty as there was no direct physical contact with patients or equipment.Pictures and videos were sourced from question bank or online and exhibited in PowerPointslides. The ease of the organization facilitated the flow of the exam within the stipulated

Figure 3.Schedule of a typicalOSCE station

QAE29,4

554

period, enhancing the process of time management significantly without much demand inadministrative requirement.

As the stations within each circuit are principally independent from one another, a largenumber of students can actually be evaluated by just constructing more concurrent circuits.Besides the additional resource for invigilation, there is no extra expenditure needed, thuscrucially reducing the arduous demands on time, cost and labour. The number of academicworkforces required to conduct the examination is notably lessened. Organizing a station isa much easier task as preparations are made within PowerPoint, as compared to thetraditional physical set up with chairs, tables and devices. The slides can be reproduced toaccommodate as many circuits as possible, with just a simple copy and paste function.

The application of Google Form as an optional method for online questions is notunheard of (Thohir, 2020). The decision to adopt Google Form for examination is based onseveral factors. Albeit its simplicity in design, the affability of the software towards users isdesirable. It is also available for free and totally compatible with any standard operatingsystem either Windows, Apple or Linux or devices as in a laptop, tablet or even asmartphone (Estan, 2018). The automaticity of the system to grade the multiple choicequestions in real time is time efficient and promotes constructive feedback from the studentsto improve the performance (Primanda et al., 2019).

In a normal scenario, conducting a professional exam requires huge financial planning.From 2017 to 2019, the cost to conduct a professional exam was approximately RM20,000per year. With the implementation of the present online exam, it does not cost even a singlecent. This platform eliminates the cost of unnecessary processes while keeping overheadexpenditure to the minimum. There is lower financial burden as the exam is conducted withhardly any physical items (Astalini et al., 2019).

Although online exam has advantages, there are limitations that must be taken intoaccount. According to the Malaysian Communications and Multimedia Commission,Malaysia’s overall internet penetration with fixed-broadband subscription is about 2.7million, with only 1.2 million above 100 mbps. Majority of the Malaysians are connected tothe internet via mobile broadband (MCMC, 2019). Slow speed and bad internet connectivitycould potentially jeopardize the fundamental process of an online system.

In the pursuit of a cheaper option, some online educational or video platforms are beingoffered for free, but at the expense of all the restrictions imposed. This hindrance may affectthe delivery of the system, such as placing a limit to the creation of questions, number ofparticipants or even duration of video call. Thus, an institution will have to considersubscribing to a platform of choice, and if possible, review on a supplemental expenditure toinvest on a stable security system to safeguard the assessment (Walsh, 2015).

The process of creating the questions may necessitate some basic computer skills, whichcould be a challenge for those who are not tech-savvy. Sometimes, the change from theoriginal format to online platform may not go as intended due to lack of specific features,and even upon successful conversion, countless repetitive trial runs need to be conducted toensure that the system executes well (Snodgrass et al., 2014). This tedious task is imperativeto identify any probable challenges.

An important element to be considered in an online exam is the functional capability ofthe software to constantly backup the information in the prevention of data loss, shouldthere be any internet connection disruption. Google Form, unfortunately, is not able to retainthe information if a user clicks on the reload button of the browser.

One of the major setback of the online OSCE is the clinical examination component,which is a task required by the candidate to perform certain physical examinationmaneuvers to look for the medical signs that reflect upon the illness. Owing to lack of

Onlineexamination

555

contact with patients, this assessment may not be plausibly carried out. We improvised theOSCEwith demonstration of the clinical technique on a simulated patient instead.

The feedback received only garnered about 42.9% of response rate. The comments fromthe students have been mostly positive, even though there are three in favour of traditionalphysical setting. All are satisfied with the exam flow and the way it was coordinatedbetween the invigilator, facilitator and candidate, agreeing that expectations are clearly met.Only two candidates express concern with lag issue, whereas three others suggest forstrategically placed toilet breaks to pace the exam. These responses are fair and accuratelyreflected on other similar online clinical exam (Kakadia et al., 2020; Major et al., 2020).

LimitationThis is an observation in a real-life scenario that discusses the purpose and the process. Asingle studymay not be sufficient to relate the reliability and generalizability of the analysis.

ConclusionConducting an online professional exam for both theory and clinical is relatively newespecially for medical institutions. There will invariably be many issues in the delivery ofthe system, but with the right technology innovation, it will be possible and perhaps bewidely used in future.

ReferencesAstalini, D., Kurniawan, W., Anwar, K. and Kurniawan, D.A. (2019), “Effectiveness of using e-module

and e-assessment”, International Journal of Interactive Mobile Technologies, doi: 10.3991/ijim.v13i09.11016.

Bunyan, J. (2020), “PM: Malaysia under movement control order from wed until march 31, all shops closedexcept for essential services”, MalayMail, available at: www.malaymail.com/news/malaysia/2020/03/16/pm-malaysia-in-lockdown-from-wed-until-march-31-all-shops-closed-except-for/1847204

Chaudhary, R., Grover, C., Bhattacharya, S.N. and Sharma, A. (2017), “Laptop assisted objectivestructured clinical examination versus objective structured clinical examination in assessmentof dermatology undergraduate students”, Indian Journal of Dermatology, Venereology andLeprology, Vol. 83 No. 4, doi: 10.4103/ijdvl.IJDVL_587_16.

Coronavirus (COVID-19) events as they happen (2020), WHO, available at: www.who.int/emergencies/diseases/novel-coronavirus-2019/events-as-they-happen#.XtMvGEZExJ8.link

Crisp, G. (2007), E-Assessment Handbook, 1st ed., Continuum International Publishing Group Ltd.Ellaway, R. and Masters, K. (2009), “AMEE guide 32: e-learning in medical education part 1: learning,

teaching and assessment”, Medical Teacher, Vol. 30 No. 5, pp. 455-473, doi: 10.1080/01421590802108331.

Estan, J.G. (2018), “Google forms for the online assessment of osces stations”, Biomedical Journal ofScientific and Technical Research, Vol. 5 No. 2, doi: 10.26717/bjstr.2018.05.0001163.

Kakadia, R., Chen, E. and Ohyama, H. (2020), “Implementing an online OSCE during the COVID-19pandemic”, Journal of Dental Education, doi: 10.1002/jdd.12323.

Kaliyadan, F., Khan, A., Kuruvilla, J. and Feroze, K. (2014), “Validation of a laptop based objectivestructured clinical examination in the assessment of undergraduate dermatology courses”,Indian Journal of Dermatology, Venereology and Leprology, doi: 10.4103/0378-6323.129386.

Major, S., Sawan, L., Vognsen, J. and Jabre, M. (2020), “COVID-19 pandemic prompts the developmentof a web-OSCE using zoom teleconferencing to resume medical students’ clinical skills trainingat Weill Cornell Medicine-Qatar”, BMJ Simulation and Technology Enhanced Learning, Vol. 6No. 6, pp. 376-377, doi: 10.1136/bmjstel-2020-000629.

QAE29,4

556

Malaysian Medical Council (2020), “Malaysian medical council guideline for conduct of onlineexamination for undergraduate medical programme during and after movement control orderdue to COVID-19 pandemic”, available at: https://mmc.gov.my/

Malaysian Qualifications Agency (2020), “Guidelines on the delivery of higher education programmesduring and post COVID-19 movement control order (MCO)”, available at: www.mqa.gov.my

MCMC (2019), “Komunikasi and multimedia: fakta dan angka terpilih, 1Q 2019 JUN 06, 2019”,Malaysian Communications and Multimedia Commission, available at: www.mcmc.gov.my/resources/statistics/communications-and-multimedia-pocket-book-of-stati

Povera, A. and Yunus, A. (2020), “Education ministry to organise home-based learning during MCO[NSTTV]”, New Straits Times, available at: www.nst.com.my/news/nation/2020/04/583180/education-ministry-organise-home-based-learning-during-mco-nsttv

Primanda, Y., Sutrisno, R. and Haris, F. (2019), “The development of online OSCE prototype for OSCEin school of nursing: lesson learned”, 10.2991/icosihsn-19.2019.47

Schmidt, S.M., Ralph, D.L. and Buskirk, B. (2009), “Utilizing online exams: a case study”, Journal ofCollege Teaching and Learning (TLC), Vol. 6 No. 8, doi: 10.19030/tlc.v6i8.1108.

Snodgrass, S.J., Ashby, S.E., Anyango, L., Russell, T. and Rivett, D.A. (2014), “Electronic practical skillsassessments in the health professions: a review”, The Internet Journal of Allied Health Scienceand Practice, Vol. 12 No. 1, pp. 1-10, available at: http://ijahsp.nova.edu/articles/Vol12Num1/pdf/Snodgrass.pdf

Tapper, J. Batty, D. and Savage, M. (2020), “Medical students take final exams online for first time,despite student concern”, The Guardian, available at: www.theguardian.com/education/2020/mar/22/coronavirus-forces-medical-students-sit-final-exams-online

Thohir, M. (2020), “Evaluation of arabic learning outcomes using google form during school quarantinedue to covid-19 pandemic”, Studia Religia: Jurnal Pemikiran Dan Pendidikan Islam.

Walsh, K. (2015), “Point of view: online assessment in medical education- current trends and futuredirections”,MalawiMedical Journal, Vol. 27 No. 2, doi: 10.4314/mmj.v27i2.8.

Corresponding authorChoon Yong Alex Tan can be contacted at: alextan@utar.edu.my

For instructions on how to order reprints of this article, please visit our website:www.emeraldgrouppublishing.com/licensing/reprints.htmOr contact us for further details: permissions@emeraldinsight.com

Onlineexamination

557