Uso de OSCEs en Evaluación

John (Jack) R. Boulet, Ph.D.Foundation for Advancement of International Medical Education and

Research

Jornadas de Educación Médica 2010“Educación por Competencias: Cambio de Paradigma”

FAIMER®

“Educación por Competencias: Cambio de Paradigma”

Universidad Nacional Autónoma de MéxicoFacultad de Medicina UNAM

February 2010

Objective Structured Clinical Examination

� Series of short stations� Candidate assessed on � Candidate assessed on

same content� Multiple evaluators� Standardization

� Conditions

FAIMER®

� Conditions� Scoring tools

OSCE� Objective

� All candidates assessed with same content and marking schememarking scheme

� Structured� Stations have a specific task or tasks� Detailed scripts for standardized patients� Candidate instructions

FAIMER®

� Clinical Examination� Measures applied clinical and theoretical

knowledge� Post-encounter exercises

OSCE Content?

� Standardized or “real” patients

� Mannequin� Part-task trainer

� Haptic models

� Computer-based

FAIMER®

� Computer-based case simulation

� Hybrid models

Assessment OSCEs in Medicine� United States Medical Licensing Examination (USMLE™)

� Step 2 CS (ECFMG Clinical Skills Assessment)� Step 3

� National Board of Osteopathic Medical Examiners� National Board of Osteopathic Medical Examiners� Medical Council of Canada� General Medical Council (UK)

� Professional Linguistics and Assessment Board� Various specialty board examinations

� American Board of Emergency Medicine� American Board of Internal Medicine

� Royal College of Physicians and Surgeons of Canada

FAIMER®

� Royal College of Physicians and Surgeons of Canada

� Dillon, G.F., Boulet, J.R., Hawkins, R.E. & Swanson, D.B. (2004). Simulations in the United States Medical Licensing Examination™(USMLE™). Quality & Safety in Health Care, 13(Suppl 1), i41-i45.

Standardized (Simulated) Patients Examinations

� Performance-based� “Standardized”� “Standardized”

� Same conditions for all test takers

� Individuals trained to portray patients with specific conditions

FAIMER®

specific conditions� Chest pain� Depression� Shortness of breath

Performance Measures� Link measures to scenario

events� Focus on observable � Focus on observable

behaviors� Incorporate multiple

measures from different sources

� Capture performance

FAIMER®

� Capture performance processes in addition to outcomes

Types of Scores� Explicit Process

� Checklists� Key actions� Key actions

� Implicit Process� Rating scales

� Timing� Sequencing

� Explicit outcome

FAIMER®

� Explicit outcome� Patient status� Complications, etc.

Why are OSCEs Important for Assessment?

� Assessment drives learning� Efficient� Efficient

� large numbers of ‘examinees’

� Provides clinical exposure� Clinical skills assessment

� Practice-based learning� Tailored feedback

FAIMER®

� Tailored feedback

� Quickly identify curricular problems

� Patient safety

Competencies Assessed with OSCEs

� Knowledge� Communication�� Counseling� Crisis management� Procedures� Physical examination� Medical decision making

FAIMER®

� Medical decision making� Leadership� Teamwork� Interviewing

Important OSCE Issues� Standardization of

conditions an content�� Choice of tasks

� Generalization

� Training� Raters, SPs

� Scoring

FAIMER®

� Scoring� QA

� Standards

Content Under-representation� Some conditions cannot be simulated very well

� Not as important for basic skills� Programming for mannequins is imperfect� Programming for mannequins is imperfect� “Real” patients can vary over time

FAIMER®

Consequences of Testing� Candidate self-selection / preparation

� want to learn� specialized preparation courses� specialized preparation courses

� Changes in curriculum� Development of training centers

FAIMER®