Assigning Milestone Evaluations in Internal

MedicineMelvin Blanchard, MD, FACP

Program Director, Internal MedicineChief, Division of Medical Education

Department of Medicine

Internal Medicine program overview ACGME Charges re Competencies Response by IM community ACGME counter response Our program’s approach to evaluation

Outline

Discipline encompassing the study and practice of health promotion, disease prevention, diagnosis, care and treatment of adults 1 of 4 physicians in the US IM residents

Our program: 151 trainees

2/3 inpatient; 1/3 outpatient BJH, VA, community, international ~50 rotations

Internal Medicine Overview

Category Positions

Number of training programs 393

Positions in match 6177

First year fellows 4584

1999 – ACGME launched the Outcomes Project Required PDs to assess trainees in 6 competencies

Patient Care Professionalism Practice-Based Learning and Improvement Interpersonal and Communication skills Medical Knowledge Systems-Based Practice

Competencies required of a physician to deliver competent medical care

ACGME Charge - 1

2009 – ACGME charged specialties with identifying milestones of competency development Observable developmental steps

How do we know that PGY-2 resident will be competent at graduation?

How do we know that graduates from the 393 IM programs can deliver same quality of IM care?

ACGME Charge - 2

2009-12: Published 142 Milestones

Aka Curricular Milestones Point in development that facilitates assessment of

progression from beginner to expected proficiency at end of training

Published 16 Entrustable Professional Activities (EPAs) KSAs critical to practice specialty

Milestones and EPAs categorized by 6 competencies

IM Community Response

Manage patients with diseases across multiple care settings. Provide age-appropriate screening and preventative care. Resuscitate, stabilize, and care for unstable or critically ill patients. Provide perioperative assessment and care.  Manage transitions of care. Facilitate family meetings. Enhance patient safety. Improve the quality of health care at individual and systems

levels. Demonstrate personal habits of lifelong learning. Demonstrate professional behavior.

EPAs

142 milestones is too numerous Not optimal format for reporting to the ACGME

ACGME and ABIM combined forces Developed milestones for reporting educational

outcomes Aka Reporting Milestones or Milestones

Grouped into 22 sub-competences Grouped into 6 competencies

With each competency associated with 2 – 5 sub-competencies

ACGME Counter Response

Date of download: 2/5/2014

Copyright © American College of Physicians. All rights reserved.

From: The Internal Medicine Reporting Milestones and the Next Accreditation System

Ann Intern Med. 2013;158(7):557-559. doi:10.7326/0003-4819-158-7-201304020-00593

Example subcompetency for systems-based practice.Copyright © 2012 The Accreditation Council for Graduate Medical Education and The American Board of Internal Medicine.

Figure Legend:

• Competencies• Curricular milestones• Reporting milestones• Milestones• Entrustable Professional Activities• Narratives • Sub-competencies

P MK PC PBLI SBP

Works in teams Cost conscious Transition of

care

Disregards communicatio

nInconsistent Recognizes

importance Uses resources Coordinates care

SBP

5 levels of milestones: Critical deficiency to Aspirational

Entrustable Professional Activities

Competencies

Competencies to Milestones

Sub-competencies

Milestones

EPAs

Curricular Milestones

What did we do?• Broke 22 sub-competencies into sub-sub competencies

– Each sub-sub competency can be used as a question on a rotation evaluation

• Rotations divided among 5 faculty– Each faculty work with core faculty/rotation directors to select

evaluation questions appropriate to rotation

• Tracked assignment to assure each sub-competency measured multiple times

• Data from conference attendance, ITE, journal club participation, etc. also feed into evaluation system

1. Gathers and synthesizes essential and accurate information to define each patient’s clinical problem(s). (PC1)

Critical Deficiencies     Ready for unsupervised practice

Aspirational

Does not collect accurate historical data.

Inconsistently able to acquire accurate historical information in an organized fashion. 

Consistently acquires accurate and relevant histories from patients. 

Acquires accurate histories from patients in an efficient, prioritized, and hypothesis-driven fashion.

Obtains relevant historical subtleties, including sensitive information that informs the differential diagnosis.

Does not use physical exam to confirm history. 

Does not perform an appropriately thorough physical exam or misses key physical exam findings.

Consistently performs accurate and appropriately thorough physical exams. 

Performs accurate physical exams that are targeted to the patient’s complaints. 

Identifies subtle or unusual physical exam findings. 

Relies exclusively on documentation of others to generate own database or differential diagnosis.

Does not seek or is overly reliant on secondary data. 

Seeks and obtains data from secondary sources when needed. 

   

Fails to recognize patient’s central clinical problems. Fails to recognize potentially life threatening problems.

Inconsistently recognizes patients’ central clinical problem or develops limited differential diagnoses.

Uses collected data to define a patient’s central clinical problem(s).

Synthesizes data to generate a prioritized differential diagnosis and problem list.

Efficiently utilizes all sources of secondary data to inform differential diagnosis.

      Effectively uses history and physical examination skills to minimize the need for further diagnostic testing.

Role models and teaches the effective use of history and physical examination skills to minimize the need for further diagnostic testing. 

Milestone distribution across Rotations

Sub-competency

1 2 3 4 5 6 7 8

Competency PC1 PC2 PC3 PC4 PC5 MK1 MK2 SBP1

Inpt Gen Med 2 3 1 1

Rheum 1 1 1 1 1

CAER 1 1 1 1

Neuro 1 1 1 2

GI consult 1

NF 1 1