MedEdWorld Updates highlight useful and easily accessible sources of information on key and important topics in medical education contributed by leading experts from around the world.

U P D AT E O NMULTIPLE CHOICE

QUESTIONSUseful resources on MedEdWorld

Dr Catherine Kennedy, AMEE Projects Offi cer

GLOSSARYMultiple Choice Questions: ‘Multiple Choice Questions (MCQs) have been a common means of written assessment in medical education for many years. They are suitable for use in areas where objectively correct answers can be ascertained, such as the basic sciences and some clinical scenarios. In general MCQs are designed with a stem – a question or scenario – and a set of responses. MCQs can be implemented in a variety of ways including:

• Single best option - a student is expected to select the best response from a range of 4-5 possible options;

• Multiple true/false questions – a student has to assess a number of statements relating to the question as either being true or false;

• Extended matching questions – students answer a series of questions relating to a scenario by selecting responses from an extended list of approximately 20 options.

MCQs have a range of advantages, such as enabling examiners to cover a large degree of content, and are increasingly easy to administer and score using computers. However, designing valid questions and responses is a demanding skill that can be time consuming, and criticism has been made that they do not assess higher-order learning and analytic skills. Further discussion of MCQs and their advantages and disadvantages can be found in our Glossary.

The MedEdWorld Glossary can be accessed at www.mededworld.org/Glossary.aspx

WEBINARSDeveloping High-quality Single- Best-Answer MCQs to Assess Application of Knowledge Using Patient VignettesPresenters: Kathy Holtzman and Dave Swanson Host Institution: National Board of Medical Examiners, USA www.mededworld.org/Webinars/Webinar-Items/Developing-High-quality-Single-Best-Answer-MCQs- t.aspx This webinar focuses on writing high-quality MCQs using patient vignettes that go beyond testing recall of isolated facts to assess application of basic science and clinical knowledge in decision making.

MedEdWorldA global online medical education community

www.mededworld.org www.amee.org

A database of the expanding vocabulary used in medical education

Hour-long interactive presentations with experts in medical education

objectively correct answers can be ascertained, such as the basic sciences and some clinical scenarios. In general MCQs are designed with a stem – a question or scenario – and a set of responses. MCQs can

• Single best option - a student is expected to select the best response from a range of 4-5 possible options;

A selection of key articles recommended by the MedEdWorld Community

TEXTBOOKSCase, S.M. and Swanson, D.B. (2002) Constructing Written Test Questions for the Basic and Clinical Sciences (3rd Ed. Rev.) Philadelphia, PA: National Board of Medical Examiners.www.mededworld.org/Publications/Textbooks/Constructing-Written-Test-Questions-For-the-Basic.aspxThe manual provides an overview of item formats, concentrating on the traditional one-best-answer and matching formats. It reviews issues related to technical item fl aws and item content; statistical indices of item quality after test administration; and standard-setting.

Haladyna, T.M. and Rodriguez, M.C. (2013) Developing and Validating Test Items. London: Routledge.www.mededworld.org/Publications/Textbooks/Developing-and-Validating-Test-Items.aspx An update and development of ‘Developing and Developing Multiple-Choice Items (3rd Ed.)’ (Haladyna, 2004), this book provides a comprehensive resource for the development and validation of multiple-choice question items and constructed response question formats.

Osterlind, S.J. (1997) Constructing Test Items: Multiple-Choice, Constructed Response, Performance and Other Formats. Boston: Kluwer Academic Publishing.www.mededworld.org/Publications/Textbooks/Constructing-Test-Items-Multiple-Choice,-Construct.aspxThis book addresses major issues in constructing test items by highlighting: the characteristics and functions of test items; editorial guidelines for writing test items in all of the commonly used item formats; methods for determining the quality of test items; procedures for ordering items in a test; item scoring schemes; and computer-generated items.

JOURNAL ARTICLESBandaranayake, R.C. (2008) Setting and maintaining standards in multiple choice examinations: AMEE Guide No. 37. Medical Teacher, 30:9-10, pp. 836-845.www.mededworld.org/Publications/Articles/Setting-and-maintaining-standards-in-multiple-choi.aspx This guide examines the more commonly used methods of standard setting, illustrates the procedure used in each with the help of an example, and discusses the advantages and disadvantages associated with the use of each.

Campbell, D.E. (2011) How to write good multiple-choice questions. Journal of Paediatrics and Child Health. 47:6, pp.322-325.www.mededworld.org/Publications/Articles/How-to-write-good-multiple-choice-questions.aspxThis paper provides a simple guideline that will assist with the production of high-quality multiple-choice papers.

McCoubrie, P. (2004) Improving the fairness of multiple-choice questions: a literature review. Medical Teacher, 26:8, pp. 709-712.www.mededworld.org/Publications/Articles/Improving-the-fairness-of-multiple-choice-question.aspxThis article reviews the literature on MCQ-testing and the perceptions that they can be ‘unfair’. The article explores these concerns before making a series of practice points for conducting fair and defensible MCQ examinations.

Markert, R.J., Goggans, D.S., and Richardson, W.S. (2012) Local Development of MCQ Tests for Evidence-based Medicine and Clinical Decision Making Can Be Successful. Teaching and Learning in Medicine, 24:4, pp. 341-347.www.mededworld.org/Publications/Articles/Local-Development-of-MCQ-Tests-for-Evidence-based.aspxThis paper describes a strategy to develop an EBM/CDM MCQ test database guided by educational theory and used psychometric analyses, including reliability, validity, and item analyses, to judge the strategy’s success.

Palmer, E.J. and Devitt, P.G. (2007) Assessment of higher order cognitive skills in undergraduate education: modifi ed essay or multiple choice questions? Research paper. BMC Medical Education, 49:7.www.mededworld.org/Publications/Articles/Assessment-of-higher-order-cognitive-skills-in-und.aspx This study was designed to assess the eff ectiveness of Modifi ed Essay Questions (MEQs) to measure higher-order cognitive skills in an undergraduate institution.

Tweed, M., Thompson-Fawcett, M., Schwartz, P., and Wilkinson, T.J. (2013) Determining measures of insight and foresight from responses to multiple choice questions. Medical Teacher, 35:2, pp. 127-133.www.mededworld.org/Publications/Articles/Determining-measures-of-insight-and-foresight-from.aspxClinical practitioners require insight, the ability to judge their personal capability, and foresight the ability to predict the potential consequences of the actions they suggest.

Zoanetti, N., Beaves, M., Griffi n, P., and Wallace, E.M. (2013) Fixed or mixed? a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program. BMC Medical Education, March 2013.www.mededworld.org/Publications/Articles/Fixed-or-mixed-a-comparison-of-three,-four-and-mix.aspxThis article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP).

RESOURCES Guidelines for writing Multiple Choice Questions (MCQs) Single Best Answer (SBA) & Extended Matching Items (EMI) formatswww.mededworld.org/Resources/Resources-Items/Guidelines-for-writing-Multiple-Choice Questions-% 28.aspx Date: 2012 Author: Intercollegiate Speciality Boards This guide provides a range of useful information for the development of MCQs, including Single Best Answer (SBA) and Extended Matching Item (EMI) formats.

LEARNING TECHNOLOGY Questionbank www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.aspx This software off ers a toolset for developing, managing and maintaining examination questions, as well as generating detailed analysis on question performance.

Questionmark www.mededworld.org/Learning-Technology/Equipment-Items/Questionmark-Enables-measurement-of-knowledge,-ski.aspx Questionmark’s technologies and services enable organizations to measure knowledge, skills, and attitudes securely for certifi cation, regulatory compliance and successful learning outcomes.

Hot Potatoeswww.mededworld.org/Learning-Technology/Equipment-Items/Hot-Potatoes.aspx The Hot Potatoes suite includes six applications, enabling you to create interactive multiple-choice, short-answer, jumbled-sentence, crossword, matching/ordering and gap-fi ll exercises for the World Wide Web.

FORUMS

MCQshttp://www.mededworld.org/Forums/f129/t4388/MCQs.aspxPost your queries, ideas and suggestions on MCQs here for discussion with colleagues.

You can also add your recommendations on topics to a forum for upload to the MedEdWorld site.

www.mededworld.org www.amee.org www.mededworld.org www.amee.org

A selection of texts recommended on MedEdWorld

Including reports, websites, video clips,images and diagrams.

Recommended to support teaching, learning and assessment

A space for debating a wide range of topics of current interest to the health professions.

The manual provides an overview of item formats, concentrating on the traditional one-best-answer and matching formats. It reviews issues related to technical item fl aws and item content; statistical indices of

www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.

This guide provides a range of useful information for the development of MCQs, including Single Best This guide provides a range of useful information for the development of MCQs, including Single Best

Post your queries, ideas and suggestions on MCQs here for discussion with colleagues.Post your queries, ideas and suggestions on MCQs here for discussion with colleagues.

www.mededworld.org www.amee.org

Copies of this and other MedEdWorld Updates are located in the Resources section of MedEdWorld. This will give you direct access to all the hyperlinks listed in the summary. Scottish Charity SCO31618

MedEdWorld was launched by AMEE to help all with an interest in health professions education learn, connect and debate key issues in medical education. Join the MedEdWorld Community by registering as a user for free or joining as a member at:

www.mededworld.org

AMEE members have full and free access to MedEdWorld. Simply login with your AMEE username and password.