Preparing the Future Primary Care Workforce Together

Primary Care Faculty Development Initiative (PCFDI)

CBME in the Ambulatory SettingNov

PCFDI(Primary Care Faculty Development Initiative)

Outline

CBME background Key concepts and definitions

Frameworks and outcomes Where we are/where we need to be

The role of milestones and entrustment in the assessment and evaluation of competence

PCFDI(Primary Care Faculty Development Initiative)

Competency versus Competent

Competency: an observable ability of a health professional, integrating multiple components such as knowledge, skills, values and attitudes.

Competent: demonstrating the required abilities in all domains in a certain context at a defined stage of medical education or practice.

Adapted from: The International CBME Collaborators, 2009

PCFDI(Primary Care Faculty Development Initiative)

What does competency-based medical education

means to you?

PCFDI(Primary Care Faculty Development Initiative)

Competency-Based Medical Education

is an outcomes-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competencies

Bottom line: CBME = Outcomes-based Medical Education

(OBME)

The International CMBE Collaborators 2009

PCFDI(Primary Care Faculty Development Initiative)

The Framework: ACGME Competencies

Medical knowledge Patient care and procedural skills Interpersonal and communication skills Practice-based learning and improvement Systems-based practice Professionalism

PCFDI(Primary Care Faculty Development Initiative)

Outcome?

PCFDI(Primary Care Faculty Development Initiative)

Frenk J, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010

The Outcome

PCFDI(Primary Care Faculty Development Initiative)

What is the outcome?

A competent (at a minimum) practitioner aligned with:

IOM Six Aims for QualityCMS Triple Aim

National Priorities Partnership

PCFDI(Primary Care Faculty Development Initiative)

Individual Physician Readiness: The Gaps

Office-based Practice Competencies Inter-Professional team skills Clinical IT Meaningful Use skills Population management skills Reflective practice and CQI skills

Care Coordination Continuity of Care Leadership and management skills Systems thinking Procedural Skills

Crosson Health Affairs 2011

PCFDI(Primary Care Faculty Development Initiative)

Is CBME/OBME Just a “Fad”?

Pet rocks Leisure suits Streaking Disco music Yugos Pokemon Tickle me Elmo

…probably not…

PCFDI(Primary Care Faculty Development Initiative)

Is CBME/OBME a Paradigm Shift?

Thomas Kuhn (1962):“Normal science, the activity in

which most scientists inevitably spend almost all of their time, is predicated on the assumption that the scientific community knows what the world is like. Much of the success of the enterprise derives from the community’s willingness to defend that assumption, if necessary at considerable cost”

Thomas S. Kuhn. The Structure of Scientific Revolutions. University of Chicago Press. Chicago. 1962. Pg. 5.

PCFDI(Primary Care Faculty Development Initiative)

Could the Same be True of UME and GME?

“Normal medical education, the activity in which most faculty inevitably spend almost all of their time, is predicated on the assumption that the medical educational community knows what the world is like. Much of the success of the enterprise derives from the community’s willingness to defend that assumption, if necessary at considerable cost”Thomas S. Kuhn. The Structure of Scientific

Revolutions. University of Chicago Press. Chicago. 1962. Pg. 5.

PCFDI(Primary Care Faculty Development Initiative)

Is CBME/OBME a Paradigm Shift?

Maybe…but perhaps that is not the main point: CBME is yet another stage on what

should be the ongoing evolution and improvement of medical education

The focus on outcomes is worthy of our attention

PCFDI(Primary Care Faculty Development Initiative)

Competency BasedEducation

Fixed length, variable outcome

Variable length, defined outcome

Structure/Process•Knowledge acquisition•Single subjective measure•Norm referenced evaluation•Evaluation setting removed•Emphasis on summative

Competency Based•Knowledge application•Multiple objective measures•Criterion referenced•Evaluation setting: DO•Emphasis on formative

Caraccio et al 2002

The Transition to Competency

Milestones

The definition of expected outcomes or competencies

PCFDI(Primary Care Faculty Development Initiative)

Milestones

A significant point in development that identifies the discrete knowledge, skills, and attitudes expected of learners as they progress through training.

Milestones should enable the trainee, program and the certification board to know an individuals trajectory of competency acquisition.

PCFDI(Primary Care Faculty Development Initiative)

Dreyfus & Dreyfus Development Model

Dreyfus SE and Dreyfus HL. A 1980Carraccio CL et al. Acad Med 2008;83:761-7

Time, Practice, Experience

Novice

Advanced Beginner

Competent

Proficient

Expert/Master

MS3MS4

PGY1

PGY3

Reporting Milestones-IM

PCFDI(Primary Care Faculty Development Initiative)

ACGMECompetency

Developmental Milestones InformingACGME Competencies

ApproximateTime FrameTrainee to Achieve Stage

Assessment Methods/Tools

Clinical skills and reasoning

Manages patients using clinical skills of interviewing and physical examination

Historical Data Gathering1. Acquire accurate and relevant history

from the patient in an efficiently customized, prioritized, and hypothesis driven fashion

2. Seek and obtain appropriate, verified, and prioritized data from secondary sources (e.g. family, records, pharmacy)

3. Obtain relevant historical subtleties that inform and prioritize both differential diagnoses and diagnostic plans, including sensitive, complicated, and detailed information that may not often be volunteered by the patient

6 months

9 months

18 months

Standardized patient

Direct ObservationSimulation

Sub-bullet

“Curricular” Milestone

PCFDI(Primary Care Faculty Development Initiative)

Milestones Benefits

Provide the learner with a clear path of progression There are no surprises

Allow for rich formative feedback. Learners know where they are and where they need to go

Define specific behaviors that can focus assessment

PCFDI(Primary Care Faculty Development Initiative)

Milestones Criticisms

Milestones are too reductionist Checklist = competence

Checking off a milestones list does not equal competent practice in a highly complex health care environment

Operationalize the milestones to develop and apply meaningful assessment and evaluation.

Entrustment/Entrustable Professional Activities (EPAs)

A framework for work-based assessment?

PCFDI(Primary Care Faculty Development Initiative)

Entrustable Professional Activities

EPAs represent the routine professional-life activities of physicians based on their specialty and subspecialty

The concept of “entrustable” means: ‘‘a practitioner has demonstrated the

necessary knowledge, skills and attitudes to be trusted to independently perform this activity.’’1

1Ten Cate O. Acad Med. 2007;82(6):542–547.

PCFDI(Primary Care Faculty Development Initiative)

An Entrustable Professional Activity

Part of essential work for a qualified professional

Requires specific knowledge, skill, attitude Acquired through training Leads to recognized output Observable and measureable, leading to a

conclusion Reflects the competencies expected… EPA’s together constitute the core of the

profession 25

ten Cate et al. Acad Med 2007

PCFDI(Primary Care Faculty Development Initiative)

“Entrustment in Medical Education”

Focused assessments around what faculty and training programs already “entrust” trainees to do? Reflects the most important outcome of

training: a trainee’s readiness to bear professional responsibility”

Enables work-based assessment focusing on demonstrating competence in desired outcomes of training.

PCFDI(Primary Care Faculty Development Initiative)

Baystate Ambulatory LMT Model

Learners: have Direct supervision Faculty member sees every patient

Managers: have Indirect supervision Faculty member discretion to see patient

Teachers: Oversight from faculty Resident discretion to allow patients to

leave before preceptingAdapted from Sudeep K. Aulakh & Michael J. Rosenblum. Presented at ICRE 2012, Ottawa.

PCFDI(Primary Care Faculty Development Initiative)

Ambulatory Milestone: Demonstrates patient-centered interviewing using the Invite, Listen, Summarize format

Failure -Frequently does not use these skills

Needs work -Inconsistently uses these skills

Competent -Consistently uses Invite & Listen; Summarizes in a reporter fashion

Proficient -Consistently uses all three skills, Summarizes interpreted information

Expert -Consistently uses all three skills, Summarizes interpreted information in complex cases

Adapted from Sudeep K. Aulakh & Michael J. Rosenblum. Presented at ICRE 2012, Ottawa.

Competencies, Milestones and EPAs

COMPETENCIES

EPAsMILESTONES

Characteristic Competencies Milestones EPAs

Granularity Low Moderate to High Low to Moderate

Synthetic/Integrated Moderate Low to Moderate High

Practicality (application)

Low Moderate High

Conceptual High Low Low to Moderate

PCFDI(Primary Care Faculty Development Initiative)

The Synthesis – Analytic Tension

Physicians do not apply each competency independently in caring for patients As a result, judging overall performance is a

synthetic/integrative assessment activity

However, You will often have to pull things apart

(analysis) to create shared mental models and to provide meaningful and actionable feedback to the resident

Lets watch a video.

What has this resident been entrusted to do?

If this were your institution, could you attest that the resident had the required

competence to provide this care?

PCFDI(Primary Care Faculty Development Initiative)

With Your Neighbor -

Pick a goal of training that would meet a national priority for ambulatory-based care.Identify two to three entrustments in training that could serve as a focus for assessment in that competency? What assessment methods would you use?What process will you use to make an entrustment decision?What facilitators and barriers would you face?

The “System”

Assessments within Program:

• Direct observations• Audit and

performance data• Multi-source FB• Patient experience• Simulation• ITExam

Judgment and Synthesis:Committee

Residents

Faculty, PDs and others

Milestone and EPAs as Guiding Framework and Blueprint

Accreditation:ACGME/RRC

NAS Milestones

Board Reporting

Program Aggregation

Certification:Board

No Aggregation

Institution and Program

PCFDI(Primary Care Faculty Development Initiative)

Break