Updates from the Residency Review Committees for Internal Medicine & Pediatrics

Updates from the Residency Review Committees for Internal Medicine & Pediatrics

MPPDA MeetingApril 10, 2014

Caroline Fischer, MBA, Executive Director, RC-PedsJerry Vasilias, PhD, Executive Director, RC-IM


What’s new…

• NAS is here… • Brief summary of NAS• Highlights from last RC meeting• Workflow Changes• Highlights from Policy and Procedures Manual• Citations and non-citations• Milestones• Recent news


One step back…

© 2013 Accreditation Council for Graduate Medical Education Information Current as of December 2, 2013

Why NAS?

• To achieve promise of outcomes-based accreditation • Annual review of programs to identify “problem

programs” to help them improve• Reduce the burden of accreditation

Some key elements of NAS:• Most data in NAS already in place• Annual ADS data entry replaces PIFs• Self-studies every 10 years• Site visited only when “issues” arise• “Internal Reviews” no longer required• Programs in good standing can innovate with “detail” PRs

• Significant % of common and specialty PRs are “detail”

• Citations, Areas for Improvement


The Data Elements of NAS…

The following are the primary annual data elements:

1) Program Attrition2) Program Changes3) Scholarly Activity4) Board Pass Rate 5) Clinical Experience Data6) Resident Survey7) Faculty Survey8) Milestones


Spotlight: Clinical Experience Data

The following are the primary annual data elements:

1) Program Attrition2) Program Changes3) Scholarly Activity4) Board Pass Rate5) Clinical Experience Data6) Resident Survey7) Faculty Survey8) Milestones


Annual Data Element #5:Med-Peds: Clinical Experience Data

• Composite variable on perceptions of clinical preparedness.

• How measured: 4th year residents’ responses to RS • IM:

• Adequacy of clinical and didactic experience in IM, subs, EM, & Neuro

• Variety of clinical problems/stages of disease?• Experience w patients of both genders and a broad age range?• Continuity experience sufficient to allow development of a

continuous therapeutic relationship with panel of patients• Ability to manage patients in the prevention, counseling,

detection, diagnosis and treatment of diseases appropriate of a general internist?


Performance Indicator # 5:Med-Peds: Clinical Experience Data• Peds:

• How well prepared are you to perform procedures without supervision? (List from PRs)

• How well prepared are you to perform patient care activities without supervision?

• How satisfied are you with the patient volume, range of patient ages, variety of medical conditions, and extent of progressive responsibility in the care of patients?

• How satisfied are you with the educational experiences to help you achieve competency in patient care skills?

• How satisfied are you with aspects of your longitudinal outpatient experience?

• Are you well prepared to competently practice general pediatrics?


Where did most of the NAS data elements come from? • In 2009, data modeling project began to identify factors

that predicted high and low program performance• Model was replicated• Data elements were assessed to determine “relative risk”

to predict low performance• Selection of Elements needed to be

• Obtainable• Meaningful• Correlates w/ prior decisions• Passed statistical “muster”• Used in combination

• Understand that this is a work-in-progress• New data elements likely in future


Role of Review Committees in NAS

• Utilize data and judgment to: • concentrate efforts on problem programs• determine whether accreditation standards are violated and

provide useful feedback for programmatic improvement• determine whether these violations (citations) rise to a level

requiring alteration in accreditation status• motivate programs to rapidly improve, rather than playing the

“accelerating accreditation action game”• over time, understand and refine the nuances of the process

• How? • Annually reviewing programs using set of data elements• Using SV as needed, and conducting complete review of the

program q10 years• Using a “PIF-less”, team based, department wide evaluation of

programs


Two steps forward….


NAS Year 1: Ground Rules

• Basic operational principle of NAS: • RCs will take an accreditation action on every program

annually.• All med-peds programs will receive notice regarding

accreditation status btw January and July.

• At January and March 2014 meeting, RCs reviewed NAS data submitted in AY 2012-2013• ADS annual update information submitted in fall of 2012• Faculty and Resident survey data from early spring of

2013• Responses to “previous citations” was current


NAS Year 1: Ground Rules

• All programs on warning or probation seen by reviewers

• All programs identified by NAS data as “troubled” underwent review by RC staff and then members• What data elements were triggered?

• Not all data elements have same importance/weight • Board scores and resident survey have more weight

– Cognizant of changes to PR for Board score pass rates for Peds

• Are programs still getting used to data elements (e.g., scholarly activity table)?

• Are there patterns/trends in data?


What did we expect would happen in NAS?


NOT this…


In OAS…82% of med-peds programs had a review cycle between 3-5 years *

* ACGME Data Resource Book 2012-2013, based on 78 med-peds programs. Book available on www.acgme.org.


NAS Conceptual Model Expected Outcomes

STANDARDS

StructureResources

Core ProcessDetailed Process

Outcomes

InitialAccreditation

New Programs

StructureResourcesCore ProcessDetailed ProcessOutcomes

Withhold AccreditationWithdrawal of Accreditation

2-4% 15% 75%

6-8%

Accreditation with Warning

New Programs,Accredited Programs with Major Concerns

Probationary Accreditation

StructureResourcesCore ProcessDetailed ProcessOutcomes

ContinuedAccreditation

Accredited Programs without Major Concerns

Continued Accreditation with

Commendation

StructureCore ProcessResourcesDetailed ProcessOutcomes


Withdrawal of Accreditation

Good Standing 75%

Withheld/Withdrawn 6-8%

Warning/Probation

15%

New (Initial) 2-4%

NAS Conceptual Model Expected Outcomes


NAS Year 1: Expected vs Actual Outcomes

NAS Projections

75%

15%

2-4%6-8%

Continued Accreditation(Good Standing)

92%)

Warning/Probation 1%

New Programs (Initial) 1%

* Site visit scheduled 6%

80 Med-Peds Residency Programs

How will NAS change the flow of information and the work of

the RC and the program?

Reporting & RC Review TimelinesAY 2012-2013

Reporting & RC Review TimelinesAY 2013-2014

What new Policies and Procedures do I need to know about ?

• Review cycles are gone! Why? Annual Accreditation.• Except for new programs, they get 2

yr cycle + a site visit • No longer “propose” adverse actions

• Probation only after site visit• Core and Subs have same status

options• Probation now an option for subs

• Citations as well as non-citations• Core + Subs re-coupled…again

• Self-study = RC will see entire dept• Probation for core = probation for

subs

Highlights from New P&P Manual


Accredited ProgramContinued Accreditation (CA)

CA w/Warning

Other (e.g. egregious)

Site Visit

Probationary Accreditation*

Withdrawal of Accreditation**

CA w/Warning

CA

* Probation cannot exceed 2 years** Does not require Probation first

Accreditation Status OptionsContinued Accreditation

Significant changes: “Continued Accreditation w Warning” appears as such on website; an adverse action no longer “proposed,” can be granted only after a site visit; subs now can also be put on “probationary accreditation.”


Citations in NAS

Citations are not new• Identify areas of non-compliance• Linked to specific requirements• Responses required in ADS • Citations are given and removed by RC (not by staff)Phase I specialties:

• Citations received in NAS (after July 1, 2013): will require an RC member to review annually.

• Citations received in OAS (given prior to July 1, 2013): will go away after two cycles of continued accreditation in NAS with no new citations.


Areas for Improvement (AFI)

AFIs are new in NAS• “General concerns”• May be given by staff (RC rules) or by RC

members • May not be specifically linked to a requirement• Do not require written response in ADS• Expectation that AFIs will be monitored locally

• PD and GMEC will work to resolve• AFIs will be tracked by RC


Citations vs AFIs

• In OAS, the main mechanism to provide feedback was through citations

• In NAS, we have 2 methods: citations and AFIs• Citations require annual review by a member of the RC

In all likelihood…citations will be used more sparingly, in hopes that AFI’s trigger appropriate

local program improvement.


Milestones


Reporting Milestones: Med-Peds• ANNUAL reporting of reporting milestones• Report BOTH IM (22) and Peds (21) milestones• Semiannual evaluation process as usual/required• Plan will be revisited

AY 2013-14• Reporting period for both IM & Peds milestones =

May 1 – June 15, 2014

AY 2014-15• Reporting period for both IM & Peds milestones =

May 1 – June 15, 2015


Reporting Milestones: Use by RCs

• De-identified, aggregate (program) data will be used as ONE data element RC can look at

• Cannot be fully used for several years• Initially, ascertain that programs are reporting• Next, check for completeness of data, etc• Review of patterns or trends will be important


Clinical Competency Committee• The PD must appoint a CCC. (Core)

• Must be composed of at least 3 faculty (Core)

• Others eligible for appointment to the committee include faculty from other programs and non- physician members of the health care team. (Detail)

• Written descriptions of responsibilities (Core)

• Review all resident evaluations semi-annually (Core)

• Prepare/assure reporting of milestones evaluations of each resident to ACGME semi-annually (Core)

• Make recommendations to the PD for resident progress, including, promotion, remediation and dismissal (Detail)


Clinical Competency Committee cont.

• PRs do not specify composition; each program may decide best structure

• PRs do not limit PD’s role• PRs do not define specialty, degree, role for

members of CCC• “Best practices” may be defined by community• Review FAQ


Recent News: Single GME Accreditation System


Other Recent News

• Faculty Development slide decks available on ACGME’s site

• Resident survey results will not be available until mid-June• ACGME leadership

aware this will be problematic for many and agreed to reassess for next yr.

Program ResourcesACGME Contacts

• Questions related to ADS: • Samantha Alvarado ([email protected]) 312.755.7118;

[email protected]

• Questions related to site visit: • Ingrid Philibert ([email protected]) 312.755.5003• Jane Shapiro ([email protected]) 312.755.5015• Penny Lawrence ([email protected]) 312.755.5014

mailto:[email protected]





Questions? • Questions related to requirements or notification letter:

• IM Staff:• Jerry Vasilias ([email protected]) 312.755.7477• Karen Lambert ([email protected]) 312.755.5785• Billy Hart ([email protected]) 312.755.5002• Jessalynn Watanabe ([email protected]) 312.755.5784

• Peds Staff:• Caroline Fischer ([email protected]) 312.755.5046• Denise Braun-Hart ([email protected]) 312.755.7478• Kim Rucker ([email protected]) 312.755.7054• Luz Barrera ([email protected]) 312.755.5077










Thank you.

“You can’t teach an old dogma new tricks.”

Dorothy Parker

Updates from the Residency Review Committees for Internal Medicine & Pediatrics

Documents

Transcript of Updates from the Residency Review Committees for Internal Medicine & Pediatrics