SES059 - Beyond Collaboration-Real-Time Integration with...
Transcript of SES059 - Beyond Collaboration-Real-Time Integration with...
SES:059 Beyond Collaboration: Real-time Integration with the C-Suite:
Using CLER, ACGME, and GME Data Innovatively to Drive Institutional Change and Improve the Clinical Learning
Environment
David Entwistle, M.A., President & CEO Stanford Health Care Ann M. Dohn, MA, DIO, Director of GMENancy Piro, PhD, Senior Program Manager
Conflict of Interest
SPEAKERS:• David Entwistle, MA ‐ ACGME Board of Directors, Member• Ann Dohn, MA – No Conflicts of Interest• Nancy Piro, PhD ‐ No Conflicts of Interest
Session Objectives
1. Better understand all the new/possible data sources DIOs, Program Directors, Hospital Leadership and Administrators can leverage for innovation
2. Understand how to integrate GME infrastructure with institutional organizational assets to effectively leverage this data collaboratively in high level institutional decision‐making e.g., resource allocation
3. Learn how to apply and fully utilize the data for collaborative change in the Clinical Learning Environment
4. Understand ways to establish integrative and collaborative processes which enable the enhancement and alignment of GME with the overall health care delivery system.
Agenda
DiscussI. The availability of multi‐sourced data and use of this data by
GME and the C‐Suite to spearhead strategic innovation in the learning environment
II. Tools that facilitate collaboration using existing data sources e.g., CLER that can integrate with C‐Suite strategic priorities
III. Specific processes that illustrate how this data can be used to collaborate with hospital leadership
Background – Historical Perspective
• Historically, many GME decisions have been made in silos devoid of any meaningful connection between:– the C‐Suite, GME, Residents, the Clinical Learning Environment (CLE)and the Hospital & Medical School mission/vision.
C‐SUITE GME/PDs
Clinical LearningEnviron‐ment
HospitalMissionVision Med School
MissionVision
Residents
Background – Different Perspectives
• The C‐Suite, GME, Residents, the Clinical Learning Environment (CLE) and the Hospital & Medical School bring different perspectives to the table, e.g.,– Service vs Education– Revenue Units vs Academic Time– UME vs GME– Patient Care vs Protected Time
• Today we have the availability of multiple data sources which can enable the “means” to facilitate this evolving partnership. ..
• Purpose– Mission, Vision, Values
• Strategies– Complex Care, Coordinated Care
• Objectives– Goals, Operating Plans, Dashboards
• Performance– Improvement and Management
Understanding C-Suite Priorities
8
STANFORD OPERATING SYSTEM…Standard Operating System…
Data Sources
GMEC‐Suite
Program
Evaluations
Program QualityEvaluations
Safety Reports & Surveys
National Patient Volumes
Benchmarking Data
ACGME RRC Accreditation Decisions
Hospital IP/OP Utilization Statistics ‐Volumes
Hospital Strategic Objectives
FinancialReports
ACGME & GME Wellness Survey
CLER Data
Patient Satisfactionassessments
ACGME & Internal Surveys
Brainstorming new data sources –what have we missed ?
Making data meaningful – speaking the C-Suite “Language”
• Applying and fully utilizing the data for collaborative change in the Clinical Learning Environment
• Establishing and maintaining a data‐based dialogue with the C‐Suite is the first step in true integration between GME and the C‐Suite.
Building Integrative and Collaborative Processes
• Setting up integrative and collaborative processes which enable the enhancement and alignment of GME with the overall health care delivery system.
MULTI‐SOURCE DATA PROCESSES COLLABORATIVE
DECISIONS
Process Example – Program Expansion and Funding
14
Historical Perspective – A3 “What was the Problem”
Understanding some of the issues surrounding C-Suite views on residency/fellowship expansion.
• SERVICE vs Education• Budget Cycle vs Academic Year• Hospital growth, expansion and strategies not tied to the educational cycle• Cost of trainees
15
Understanding some of the issues surrounding Program Views on Residency/fellowship Expansion.
• EDUCATION vs Service• Newly Recognized Areas of Training…
– Epilepsy– Peds Epilepsy– Neuroimmunology– Selective Pathologies
• Faculty recruitment / retention‐I want a fellow to “teach” …”
16
How Did We Design a New Process?
First Steps • Talked with Stakeholders to determine their current and future needs and issues.
– Discussions with C‐Suite, Dean, Chairs, DFAs (Department Business Managers), GMEC, Program Directors and Program Coordinators.
• Aligned the educational needs and issues with Institutional Deadlines– ACGME requirements for funding guarantees– NRMP Quota Deadlines
• Preliminary Process Designed …tweaked and retweaked• Debriefed new process with GMEC
17
Timeline
• Application materials/requests due to GME – July 1st for September Meeting• Second Biannual meeting – Requests due to GME October 1st for January Meeting
18
• Members:– CEO and President, Stanford Health Care (or designee)– Dean, Stanford Medicine (or designee)– Associate Dean GME, Chair GMEC– DIO, Director of GME– COO, Stanford Health Care– CMO, Stanford Health Care– Two Department Chairs– Two GMEC resident reps (1 yr term)– One (1) Program Coordinator
New Program Expansion/Funding GMEC Subcommittee
Looking into the Data:
20
Program Expansion / Funding Dashboard with Multi-Source Data
21
Determining the need for special program reviews
– More data required– Program too small– Long interval since last formal review
22
• Results of the special review if applicable• Focus on educational quality of programs• Alignment with hospital mission/strategic plans• Interval growth in teaching faculty, specialty institutes and departments that offer educational opportunities
• Long term plans for departmental growth• Program specific growth in ambulatory and inpatient volume (historical and projected)• Workforce needs
Discussion Points for GMEC Expansion/Funding Subcommittee
Decision Making Rationale
24
Summary of Program Expansion Committee Results
Decision Making RationaleResults of Funding Decisions
GME/CEO Collaborative Expansion/Funding Process Points
• Subcommittee recommendations go to the GMEC for final approval.• Positions are not granted or funded outside this process.• The process is transparent with results immediately posted on the GME website.
Real transparency isn’t this !!!
28
If Funding is Denied: Programs Can Re-apply for Funding
Deny
Program Expansion/Funding Committee Decision
Can re‐apply after two years but decision will again be based on same parameters including
educational quality and priorities
“The course of true love never did run smooth….”William Shakespeare
• What would we do differently?– Align Meetings with NRMP / Recruitment Deadlines– Increase communication among all the stakeholders during process development– Put final process on website sooner
30
Maintain Open Communication
Expansion FundingProcess
Program Directors
Dean
C‐SuiteVPs
DEPTChairsDFAs
31
Discussion – How is this accomplished at your institution? What would you like to change?
Part Two: Using CLER Results to Improve our Clinical Learning Environment
CLER Example – Integrating GME into the Institutional Culture of Safety – Breaking Down the Silos
Why the Clinical Learning Environment is so important...
• Essential to:– Training of Medical Students, Residents & Fellows to become competent physicians
• Trainee Supervision• Trainee Wellness, Fatigue Management & Mitigation• Trainee Professionalism
– Patient Care, Healthcare Quality and Safety• Effective Transitions of Care• Addressing Healthcare Disparities
Culture of Safety
• GME is pivotal to the institutional safety culture …• Residents are imperative to the overall safety culture…
Do we have any data from residents that will inform GME and the C‐Suite?
The answer is “CLEAR”…..e.g. CLER !
• First CLER Site Visit – provided us with a qualitative report – what to do with it?• Second CLER Site Visit – provided us with another qualitative report
– We attempted to analyze trends between the first/second reports to see if our efforts were working, but it was challenging:
• the majority of data was narrative• some questions changed• Multi‐site comparisons weren’t possible• Lack of normative data
• Next Steps: Transform CLER data to a format that would enable collaboration with hospital leadership to implement C‐Suite strategic priorities.
But just HOW can we do that?
Building Integrative and Collaborative Processes
• Establish integrative and collaborative processes which enable the enhancement and alignment of GME with the overall health care delivery system.
MULTI‐SOURCE DATA PROCESSES COLLABORATIVE
DECISIONS
We had CLER data… but how do we transform it to make it meaningful ?
Quantification of Narrative CLER findings … Intra Institutional Trends
CLER AREAS‐Questions
= No Response
Residents47
Faculty55
PDs44
Residents51
Faculty56
PDs47
Patient SafetyKnew Hospital and Clinics’ priorities in the area of patient safety
75% 79% 93%
Experienced an adverse event or near miss at
75%
Had the opportunity to participate in a patient safety investigation such as an RCA (PGY3 and above residents)
30% 64% 66%
Had personally reported a patient safety event at Medicine using the hospital’s reporting system
53% 25% 15%
Believe their residents and fellows report the event using the hospital’s reporting system
59% 52%
Believe their residents and fellows rely on a nurse to submit the report
18% 13% 7%
Believe their residents and fellows / or rely on their supervisor to submit the report
29% 20% 35%
Believe their residents and fellows care for the patient but do not / chose not submit a report
10% 9% 7%
Have reported a near miss at Medicine in the past year
29%
Received feedback on the outcome of a report that had been filed by any means.
28%
Participated in an interprofessional safety investigation (PGY 3 and above) .
38%
Have received education about proactive risk assessments (e.g., Failure Mode and Effects Analyses)
45%
Spring 2017Summer 2014
STEP 1: Identify each reported area and its respective percentages by resident, faculty or PD group
Normative data (Safety as an example) from CLER reports
STEP 2: Add National Normative data from ACGME
Take Aways
• Identified the need to improve resident involvement in Safety initiatives• Formed a Resident Safety Council – Good initial results…
Displaying normative data from CLER to monitor trends to assess the success of continuous quality improvement efforts.
STEP 3: Test significance to monitor progress on our initiatives
Understand how to use normative data from CLER to monitor trends to monitor the success of continuous quality improvement efforts together with tests of significance
• Step 4: Color Code your data Red/Green with respect to National Normative Data
CLER AREAS‐QuestionsSignificant Difference (p < .05) Direction of Difference NORM
Q Number
Population Positive NegativeChi‐
squarep‐value Z‐Score p‐value Residents Positive Negative Faculty PDs
Healthcare QualityHad participated in QI project of their own design or one designed by their program or department (PGY2s and above)
Significant Difference/Stanford Higher than Norm
76% B9 8298 6306 1992 7.223 p<.01 2.688 0.007 91% 52 5 NR NR
Project was in some way linked to the hospital’s goals Not Significant
52% B10 6154 3200 2954 3.069 p>.05 ‐1.752 0.08 40% 23 34 NR NR
Didn't know if project was in some way linked to the hospital’s goals
40% NR NR
Engaged in inter‐professional QI teams working on performance improvement projects
Significant Difference/Stanford Lower than Norm
75% B11 3133 2350 783 6.994 p<.01 ‐2.645 0.008 59% 34 23 NR NR
Residents have access to organized systems for collecting and analyzing data for the purpose of QI Not Significant
63% B12 8549 5386 3163 0.089 p>.1 0.298 0.77 65% 37 20 77% 82%
Knew the hospital’s priorities with regard to healthcare disparities Not Significant
55% B13 8607 4734 3873 0.389 p>.1 ‐0.624 0.53 51% 29 28 45% 60%
Transitions in CareKnew the hospital’s priorities for improving transitions of care
86% 92% 98%
Used a standardized process for sign‐off and transfer of patient care during change of duty Not Significant
88% B14 8471 7454 1017 0.004 p>.1 ‐0.064 0.95 88% 50 7 NR NR
If a standardized process is used, written templates of patient information are used to facilitate the hand‐off process.
Significant Difference/Stanford Lower than Norm
78% B15 7262 5664 1598 7.227 p<.01 ‐2.688 0.007 64% 36 21 NR NR
201457 Residents
55 Faculty Members44 PDs
2014ACGME Norm Data
Quantification of CLER findings across External Institutions in a Scorecard Format
CLER AREAS‐Questions
2014ACGME Norm Data
NORMResidents
47Faculty55
PDs44
Residents51
Faculty56
PDs47
Residents 53 Faculty 37 PDs 31 Residents 58 Faculty 55PDs 39
Residents12
Faculty 13
PDs 4
Residents12
Faculty 17
PDs3
Patient SafetyKnew Hospital and Clinics’ priorities in the area of patient safety
74% 75% 79% 93% 64% 58% 68% 100% 100% 100%
Experienced an adverse event or near miss at
68% 75% 77% 71% 75% 100%
Had the opportunity to participate in a patient safety investigation such as an RCA (PGY3 and above residents)
30% 64% 66% 0% 62% 100%
Had personally reported a patient safety event at Medicine using the hospital’s reporting system
47% 53% 25% 15% 20% 34% 24% 23% 25% 33% 92% 50% 66%
Believe their residents and fellows report the event using the hospital’s reporting system
59% 52% 56% 59% 47% 33% 33% 77% 50% 8% 88% 100%
Believe their residents and fellows rely on a nurse to submit the report
18% 13% 7% 18% 8% 7% 20% 11% 18% 11% 0% 0%
Believe their residents and fellows / or rely on their supervisor to submit the report
29% 20% 35% 45% 31% 21% 29% 38% 46% 56% 23% 50% 12%
Believe their residents and fellows care for the patient but do not / chose not submit a report
10% 9% 7% 12% 6% 14% 17% 4% 3% 0% 0% 0% 0%
Have reported a near miss at Medicine in the past year
20% 29% 13% 17% 50%
Received feedback on the outcome of a report that had been filed by any means.
47% 28% 35% 31% 38% 25%
Participated in an interprofessional safety investigation (PGY 3 and above) .
41% 38% 53% 57% 83% 43% 69% 51% 50% 82% 100%
Have received education about proactive risk assessments (e.g., Failure Mode and Effects Analyses)
45% 35% 41%
Have received training on how to disclose medical errors to patients and families
51% 66% 92%
If they were to be involved in a major patient safety event resulting in a patient 92% 93% 100%
Institution "A"Spring 2017
Institution "C"Winter 2014
Institution "C"Fall 2015
Institution "A"Summer 2014
Institution "B"Winter 2015
Institution "B"Summer 2017
How can all this CLER data be used to make changes in the Clinical Learning Environment at your institutions
• Investigate and leverage CLER Trends, Significance and Normative Data– Focus on areas where the institution is lower than:
• the ACGME CLER National Norms• comparative institutions
– Trending changes across CLER visits• Input to the AIR (ACGME Required Annual Institutional Review)
Analysis of the Effectiveness of our Initiative
• This full integration also engages our residents and fellows in the design of healthcare organizations that will deliver care across a complex continuum of care.
• Collaborative real‐time integration will concomitantly promote ACGME’s overall vision to support rethinking our healthcare model to enduring institutions.
Going beyond CLER …
• What else do we know about the Clinical Learning Environment?– A wealth of data coming in ….
• CLER Reports• GQ (Graduation Questionnaire – Medical Students)• Internal Resident Surveys• ACGME Resident and Faculty Surveys• Resident and Faculty Program Evaluations• Trainee Alumni Surveys• ACGME Accreditation Reports• LCME Reports
What do we know about the Clinical Learning Environment?A wealth of data coming in ….
Clinical Learning
Environment
CLER Reports
GQ
ACGME Resident and
Faculty Surveys
Internal Reviews & Resident Surveys
Trainee Alumni Surveys
LCME Reports
ACGME Accreditation
Reports
Trainee Alumni Surveys
Resident and Faculty Program
Evaluations
Group Exercise
• Discuss how the increased availability of multi‐sourced data facilitates innovative collaboration between GME, PDs and the C‐Suite spearheading strategic changes in the learning climate. (15 minutes – interactive)
All these data sources enable further discussion and collaboration between GME and the C-Suite
Questions