Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ......
Transcript of Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ......
![Page 1: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/1.jpg)
Going the Extra MILE in Your Quality Improvement Curriculum:
Integrating QI Assessment Tools and the Pediatric Milestones
Mackenzie S. Frost, MD Mario Cruz, MD
Ashley Lucke , MD Jennifer Walsh, MD
![Page 2: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/2.jpg)
Outline
• The Milestones Project – Introduction – QI Specific competencies
• Quality Improvement Curriculum – QI evaluation tools – How do we use them to assess competencies? – Future directions
![Page 3: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/3.jpg)
The Big Question
How do we evaluate quality improvement participation for the
Pediatric Milestone Project?
![Page 4: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/4.jpg)
The Milestone Project
• Redefines the evaluation framework for trainees – Add critical subcompetencies not explicit in
original ACGME competencies – Refine the competencies in the context of each
specialty – Identify or develop tools for assessment of
performance
![Page 5: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/5.jpg)
Framework for Assessment
Knows
Understands
Basic knowledge
Knows How Shows Demonstrates
Competence
(Workshop/ Simulation)
Does
Demonstrates in actual practice Performance
Applies
Integrated knowledge
Knowledge Attitudes Skills
Miller GE. The assessment of clinical skills/competence/ performance. Academic Medicine, 1990;65:563–567
![Page 6: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/6.jpg)
Novice Expert
![Page 7: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/7.jpg)
QI Subcompetencies Pediatrics
Problem-Based Learning 3 (PBLI3): Systematically analyze practice
using quality improvement methods, and implement changes
with the goal of practice improvement
System-Based Practice 2 (SBP2): Advocate for quality patient care and optimal patient care systems
System-Based Practice 3 (SBP3): Work in inter-professional teams
to enhance patient safety and improve patient care quality
![Page 8: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/8.jpg)
QI Subcompetencies Pediatric Subspecialties
Problem-Based Learning 4 (PBLI4): Systematically analyze practice
using quality improvement methods, and implement changes
with the goal of practice improvement
System-Based Practice 3 (SBP3): Incorporate considerations of cost
awareness and risk-benefit analysis in patient and/or population based care as
appropriate
System-Based Practice 5 (SBP5): Work in inter-professional teams
to enhance patient safety and improve patient care quality
System-Based Practice 6 (SBP6): Participate in identifying system
errors and implementing potential system solutions
How do we evaluate these competencies and milestones?
![Page 9: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/9.jpg)
Large Group Discussion
How are you currently assessing your quality
improvement curriculum?
![Page 10: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/10.jpg)
Quality Improvement Educational Practices in Pediatric Residency Programs: Survey of
Pediatric Program Directors • Describe current QI educational practices,
evaluation methods, and program director perceptions
• 46 question online survey with skip logic – Curricular design and content – Curriculum support – Program evaluation – PD perspectives
• Response rate 53% (104 of 197)
Mann, K. et al. Academic Pediatrics. 2014;14:23-28.
![Page 11: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/11.jpg)
Quality Improvement Educational Practices in Pediatric Residency Programs: Survey of
Pediatric Program Directors • QI education programs present at 85% of
pediatric residency programs – 83% contained didactics or formal lectures – 88% required project participation
• Clinical care • Hospital operations • Residency program • Personal performance
Mann, K. et al. Academic Pediatrics. 2014;14:23-28.
![Page 12: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/12.jpg)
Evaluation Within QI Curricula
• Participant Satisfaction – Individual lectures 32% – Entire curriculum 25%
• Self assessment of QI proficiency 24% • Knowledge 11% • Formal scoring of QI project 22% • Patient Outcomes 35% • No evaluation 17%
Mann, K. et al. Academic Pediatrics. 2014;14:23-28.
![Page 13: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/13.jpg)
Kirkpatrick’s Evaluation of Training QI Curriculum
Kirkpatrick, DL, Evaluation of training. In: Craig, R, et.al, ed. Training and Development Handbook. NY, NY: McGraw-Hill; 1967:87-112
Level 1: Reaction Attitudes
Level 2: Learning Knowledge
Level 3: Behavior Process Measures
Level 4: Results Outcome Measures
Level 5: Cost Effectiveness
![Page 14: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/14.jpg)
QI Evaluation Tools
MERIT QIPAT-7
QIKAT
![Page 15: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/15.jpg)
MERIT
Mayo Evaluation of Reflection on Improvement Tool
• Residents must be able to critically reflect on events in practice in order to develop meaningful QI interventions
• Residents kept improvement logs – Focused on clinical events – Evaluated with MERIT
Wittich, CM, et.al, Validation of a method to measure resident doctors’ reflection on quality improvement. Med Ed 2010. 44:248-55.
![Page 16: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/16.jpg)
MERIT
Wittich, CM, et.al, Validation of a method to measure resident doctors’ reflection on quality improvement. Med Ed 2010. 44:248-55
Problem of Merit
Reflection on Personal
Characteristics of QI
Reflection on System
Characteristics of QI
![Page 17: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/17.jpg)
MERIT
Problem of Merit • Event was patient centered • Potential for event to effect other patients • Event could cause negative clinical impact • Overall problem of merit • Event was evidence based in its description • Overall improvement opportunity
Wittich, CM, et.al, Validation of a method to measure resident doctors’ reflection on quality improvement. Med Ed 2010. 44:248-55
![Page 18: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/18.jpg)
MERIT
Reflection on Personal Characteristics of QI • Resident questioned personal practice •Quality of reflection • Contributing personal factors identified • Sufficient details to delineate contributing factors •Multiple options for personal change considered • Relevant new behaviors proposed •Next steps towards personal change considered
Wittich, CM, et.al, Validation of a method to measure resident doctors’ reflection on quality improvement. Med Ed 2010. 44:248-55
![Page 19: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/19.jpg)
MERIT
Reflection on System Characteristics of QI • Quality of reflection on institution/health care
system • Current institutional practice/system questioned • Contributing system factors identified • Multiple options for system change considered • Relevant changes to system proposed • Next steps towards system change identified
Wittich, CM, et.al, Validation of a method to measure resident doctors’ reflection on quality improvement. Med Ed 2010. 44:248-55
![Page 20: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/20.jpg)
![Page 21: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/21.jpg)
QIPAT-7
• Mayo Internal Residency Program – 144 residents participating in QI curriculum – Each month, 4 pairs of residents create QI
proposal • QIPAT-7 tool developed to evaluate the
proposals • Initial tool tested and adapted • Final tool pilot tested and raters discussed
scoring variations
Leenstra, JL, et.al, Validation of a method for assessing resident physicians’ quality improvement proposal. Journal Gen Int Med. 2007;22(9):1330-34.
![Page 22: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/22.jpg)
QIPAT-7
QI Proposal Assessment Tool
– Definition of problem – Identification of key stakeholders – Evidence of root cause analysis – Choice of QI project – Potential interventions – Proposed intervention – Implementation and evaluation
Leenstra, JL, et.al, Validation of a method for assessing resident physicians’ quality improvement proposal. Journal Gen Int Med. 2007;22(9):1330-34.
![Page 23: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/23.jpg)
Leenstra, JL, et.al, Validation of a method for assessing resident physicians’ quality improvement proposal. Journal Gen Int Med. 2007;22(9):1330-34.
QIPAT-7
![Page 24: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/24.jpg)
QIKAT Quality Improvement Knowledge Application Tool
• Assesses application of QI principles to clinical cases – Aim – Appropriate measures – Proposal for change via a potential QI project
• Designed for Internal Medicine residents
Morrison, L, et.al, The quality improvement knowledge application tool: An instrument to assess knowledge application in practice-based learning and improvement. Journal of General Internal Medicine. 2003;18:250
![Page 26: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/26.jpg)
Pediatric QIKAT
• Version #1 (2010-2011) – Poor inter-rater reliability
• Revised (Spring 2012)
• Version # 2 – Face validity, content validity
– Cincinnati Children’s AIM Course directors
– Further validation by 4 pediatric residency programs
![Page 27: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/27.jpg)
Pediatric QIKAT #2
![Page 28: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/28.jpg)
Both Pediatrics/Subspecialites
Problem-Based Learning 3/4 (PBLI 3/4)
Systematically analyze practice using quality improvement methods, and implement changes with the goal of practice improvement
![Page 29: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/29.jpg)
MERIT QIKAT, QIPAT-7
![Page 30: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/30.jpg)
Small Group Activity
Use QI assessment tools to evaluate provided QI
projects/scenarios
![Page 31: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/31.jpg)
Quality Improvement Evaluation Tools Overview
QIKAT
Quality Improvement Knowledge Application Tool
•Use - Assesses application of basic QI principles to clinical cases
•Scoring – •Pediatric QIKAT – answer
sheet outlines possible correct responses for each question
QIPAT -7 QI Proposal Assessment Tool
•Use – Assesses quality Improvement project proposals; evaluates 7 elements of the proposal
•Scoring – Scale of 1 – 5. •1 = needs improvement •3 = meets expectations •5 = exceeds expectations.
•All descriptive anchors for each project element must be met in order to score a 3.
MERIT
Mayo Evaluation of Reflection on Improvement Tool
•Use – Evaluates resident reflection on clinical events written in an improvement log
•Scoring – Scale of 1 – 4. •1 = “no” or “bottom
quartile” •2 = “somewhat” or
“second quartile” •3 – “almost” or “third
quartile” •4 – “yes” or “top quartile”
![Page 32: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/32.jpg)
Assign your trainee to a milestone level
![Page 33: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/33.jpg)
Large Group Discussion
What were the pros and cons of the tool your
group used?
![Page 34: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/34.jpg)
Resident Reflection/MERIT
QIKAT QIPAT PBLI 3/4
Intern A Extubation in NICU during kangaroo care, good system but no personal reflection
Doesn’t know SMART aim, balancing measures Score 8/16
PICU parent-staff communication, good AIM statement, lacks specific intervention and timeline
Level 2
Senior B Infant with NAS, good system and personal reflection
Well done Score 16/16
NICU UVC placement confirmation Level 3.5
Intern C Missed positive UTI culture in clinic, good personal reflection but no system reflection
Doesn’t know SMART aim, balancing measures Score 8/16
Seizure education in ED, aim statement isn’t SMART, needs more background, stakeholder, systems evaluation
Level 2
Senior D Dehydration in infant leading to PICU admit, good system and personal reflection
Well done Score 16/16
Problem list use in clinic, some issues with aim statement, good timeline
Level 3.5
![Page 35: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/35.jpg)
MERIT at UTSW
• Added as evaluation for 2014-2015 academic year – Pediatric Residency – Neonatal Fellowship
• Took advantage of existing systems – Residency – MedHub – Fellowship – Biannual IDP
![Page 36: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/36.jpg)
![Page 37: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/37.jpg)
UTSW QI Curriculum Timeline
PL1
• MERIT reflection Assessment #1 via MedHub – block 4 and 5 • IHI Courses (second half of year) - blocks 10, 11, 12 • MERIT reflection Assessment #2 – block 13, block 1 (PL2)
PL2
• Scholarly Project in Quality Improvement • Additional IHI Courses • MERIT reflection Assessment?
PL3
• MERIT reflection Assessment #3 (all third years) – block 4 and 5 • QI Scholarly Project Presentation • Project Evaluation
![Page 38: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/38.jpg)
UTSW QI Curriculum Timeline
PL1
• MERIT reflection Assessment #1 via MedHub – block 4 and 5 • IHI Courses (second half of year) - blocks 10, 11, 12 • MERIT reflection Assessment #2 – block 13, block 1 (PL2)
PL2
• Scholarly Project in Quality Improvement • Additional IHI Courses • MERIT reflection Assessment?
PL3
• MERIT reflection Assessment #3 (all third years) – block 4 and 5 • QI Scholarly Project Presentation • Project Evaluation
![Page 39: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/39.jpg)
![Page 40: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/40.jpg)
Great… What about the other
sub competencies?
![Page 41: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/41.jpg)
Pediatric Subspecialties
System-Based Practice 6 (SBP6):
Participate in identifying system errors and implementing potential system solutions
![Page 42: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/42.jpg)
QIKAT, QIPAT-7 MERIT
![Page 43: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/43.jpg)
Participate in identifying system errors and implementing potential system solutions
• Ideas: – Participate in Hospital-wide system-focused M&M – Participate in microsystem root cause analysis – Work with hospital QI/PI teams in project
development
• But how do you evaluate the ability to discuss
and acknowledge errors?
![Page 44: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/44.jpg)
Pediatric Subspecialties
System-Based Practice 3 (SBP3):
Incorporate considerations of cost awareness and risk-benefit analysis in patient and/or population based care as appropriate
![Page 45: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/45.jpg)
![Page 46: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/46.jpg)
![Page 47: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/47.jpg)
Evaluating Cost Awareness and Risk-Benefit Analysis
• New curriculum under design for addition into QI curriculum – Goal 2015-2016
• Didactic and exercise/project components – Evaluation tool for project? – Area for study and publication
![Page 48: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/48.jpg)
Both Pediatrics/Subspecialites
System-Based Practice 5 (SBP5):
Work in inter-professional teams to enhance patient safety and improve patient care quality
![Page 49: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/49.jpg)
![Page 50: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/50.jpg)
Validated Evaluation Tools for Teamwork
Self Assessed Team Skills
• Rochester Communication Rating Scale
• Team Skills Scale • Leadership and
Group Development Assessment
• ALERT questionnaire
• Team Development Wheel
• Team Dimensions Rating form
Self Assessed Attitudinal Changes
• Human Factors Attitude Survey
• Attitudes Towards Heath Care Rating Scale
• Readiness in Interprofessional Learning
Teamwork impact on quality
• None
Chakraborti et al. A Systematic Review of Teamwork Training Interventions in Medical Student and Resident Education. J Gen Intern Med 23(6):846-53
![Page 51: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/51.jpg)
Pediatrics
System-Based Practice 2 (SBP2):
Advocate for quality patient care and optimal patient care systems
![Page 52: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/52.jpg)
![Page 53: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/53.jpg)
Patient Tracer Activity • Objectives/strategies: • Observe patient encounters in the sick clinic, primary pediatrics clinic, and
ambulatory pediatric clinics paying particular attention to: – Patient wait times – Response to patients with limited English proficiency – Response to patients with limited health literacy – Response to parental frustration – Overall customer service
• Identify three aspects of the overall patient experience in which our clinics
perform better than you would have expected • List three aspects of the overall patient experience in which our clinics perform in
a manner that are (in your opinion), below acceptable • Describe three quality improvement projects that could be implemented to
improve the overall quality of the patient experience within the CCAH outpatient practices.
![Page 54: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/54.jpg)
Keeping Tabs on Resident QI Activities
This workshop is so
great.
I agree, but I wish I knew more about how other programs keep
tabs on the QI activities of their
residents?
Yeah, I mean, how can you assign milestones to QI activities if you
don’t know what activities your
residents are engaged in?
![Page 55: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/55.jpg)
St. Chris experience with QI milestones
• All resident activities are documented at semi-annual meetings with PD’s
• QI milestones assigned by a subset of three faculty – Leader of QI Curriculum – Leader of Advocacy Curriculum – Associate Program Director
• Key questions: – Does the activity involve non-physicians? – Does the activity have a hospital or community wide
impact? – Does the resident have a leadership role? – Will the results be disseminated?
![Page 56: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/56.jpg)
Name of Activity Activity Type
Name of Resident
Does project include non-physicians on
the team?
Hospital or community-
wide impact?
Leadership role on this
activity?
Scholarly dissemination
Curriculum Committee Committee Anderson Cooper No Yes No No
Global Health Committee Committee Anderson Cooper No No Yes No
Inpatient BMI Project QI
Anderson Cooper Yes Yes Yes Yes
Wellness Committee Committee Anderson Cooper Yes Yes Yes Yes
Hospital BMI group QI Bill O’Reily Yes Yes Yes Yes
Vitamin D Study Research Jessica Simpson No No Yes Yes Improving Efficiency of Family-Centered
Rounds QI Jessica Simpson Yes Yes No No Pulmonary HTN
research Research Eli Manning No No No Yes
Wellness Committee Committee Mike Tyson Yes Yes Yes Yes Improving
Debriefings after Patient Death QI Mike Tyson Yes Yes Yes No
Improving Efficiency of Family-Centered
Rounds QI Mike Tyson Yes Yes No No
![Page 57: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/57.jpg)
Large Group Discussion
How can we better evaluate the competencies
related to QI?
![Page 58: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/58.jpg)
Please Evaluate Us!
• Paper evaluations
• https://www.appd.org/amsurvey/
• Thank you for participating!
![Page 59: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/59.jpg)
QIPAT-7 Items Mean SD ICC 95% CI Cronbach’s
α Definition of the problem
3.4 0.995 0.79 0.61, 0.84
Identification of stakeholders
2.6 1.116 0.91 0.84, 0.94
Root Cause analysis
2.3 1.05 0.93 0.87, 0.95
Choice of QI Project
3.4 1.08 0.82 0.62, 0.84
Potential Interventions
2.1 0.968 0.87 0.78, 0.91
Proposed Interventions
2.9 1.1 0.85 0.65, 0.86
Implementation and Evaluation
1.9 1.12 0.9 0.82, 0.93
0.87 Leenstra, JL, et.al, Validation of a method for assessing resident physicians’ quality improvement proposal. Journal Gen Int Med. 2007;22(9):1330-34.
![Page 60: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/60.jpg)
MERIT
Wittich, CM, et.al, Validation of a method to measure resident doctors’ reflection on quality improvement. Med Ed 2010. 44:248-55
![Page 61: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/61.jpg)
QIKAT
• Study of 1 month PBLI curriculum for internal medicine residents – Administered QIKAT pre and post curriculum
Ogrinc G et al. Teaching and Assessing Resident Competence in Practice-based Learning and Improvement. J Gen Intern Med. May 2004; 19(5 Pt 2): 496–500
Pretest Mean (SD)
Post test Mean (SD)
P value
Participants 9.2 (2.6) 11.4 (2.4) <0.5
Non Participants
8.2 (2.2) 8.7 (2.5) NS
P Value NS < 0.1
![Page 62: Milestones and Quality Improvement - APPD · PDF fileGoing the Extra MILE in Your Quality ... •ALERT questionnaire •Team ... Milestones and Quality Improvement](https://reader031.fdocuments.in/reader031/viewer/2022030411/5a9dd84e7f8b9a96438d33bd/html5/thumbnails/62.jpg)
Example Key Driver Diagram
IMPROVE EFFECTIVENESS
OF QI CURRICULUM
SMART AIM
INTERVENTIONS KEY DRIVERS
An effective tool for breaking down complex questions or improvement goals and structuring them into smaller, more focused “drivers”
By July 2015, we will implement one QI evaluation tool to inform reporting in December 2015 in our residency QI curriculum
Have an evaluation tool
Buy-in for use of tool (resident and faculty)
Identify faculty who will use the tool; location for use
Identify milestones to assess
PBLI2
MERIT (milestone PBLI2 levels 1-3)
Include in goals/objectives for curriculum
Location—PL1s can complete in continuity clinic ; plan time during conference