Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding...
Transcript of Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding...
![Page 1: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/1.jpg)
Understanding the Outcomes Research Toolbox
Justin B. Dimick, MD, MPH Henry King Ransom Professor of Surgery Director, Center for Healthcare Outcomes & Policy University of Michigan ACS Outcomes Research Course 2014
![Page 2: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/2.jpg)
Agenda:
1. Conceptual overview – three major areas of research
2. For each area, explore major
problems and tools for fixing them
![Page 3: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/3.jpg)
Comparative effectiveness
Randomized clinical trials
Quality of life
? Qualitative research
Patient centered outcomes
Decision analysis
Policy evaluation
Disparities
![Page 4: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/4.jpg)
How can we make sense of these different research areas?
Groups of 3 to 4
5 minutes
![Page 5: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/5.jpg)
Evaluating Disease Management (Patient-Level)
Healthcare Delivery System (Macro-systems)
Understanding Local Provision of Care (Micro-systems)
Patient-Centered Outcomes Comparative Effectiveness
Data Synthesis
Quality Improvement Patient Safety
Policy Evaluation Payment Reform
Work Force
Coverage Decisions Technology Adoption
Cost Effectiveness
Implementation Science
Quality Measurement
![Page 6: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/6.jpg)
Problems & Solutions
![Page 7: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/7.jpg)
Evaluating Disease Management(Patient-Level)
Patient-Centered OutcomesComparative Effectiveness
Data Synthesis
Major Problems?
![Page 8: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/8.jpg)
VS.
![Page 9: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/9.jpg)
VS.
Problem: Patients getting treatment A and B are different
![Page 10: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/10.jpg)
![Page 11: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/11.jpg)
Treatment (Endo vs. open
aneurysm)
Outcome (1-year death)
Confounding variables
(Illness levels)
Confounding
![Page 12: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/12.jpg)
Selection bias & confounding
Toolbox: • Randomized clinical trials • Multivariate adjustment using
regression • Propensity scores • Instrumental variable analysis
![Page 13: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/13.jpg)
Problem: External Validity
![Page 14: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/14.jpg)
Randomized Clinical Trials – External Validity?
Toolbox: Pragmatic trials Cluster randomized trials Observational studies
![Page 15: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/15.jpg)
CER Methodologies
Randomized trials
Observational Studies
Pragmatic Trials Cluster Randomized Trials Natural Experiments Cohort
*Lohr, Medical Care (2007) 45 (10); S5
Problems with external validity
Problems with confounding
![Page 16: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/16.jpg)
What about the patient perspective?
Toolbox: Patient centered outcomes; Quality of life measures; Survey research
Dr: “Congratulations, your hernia if fixed” Patient: “Great, but why is my groin pain worse?”
![Page 17: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/17.jpg)
What about the patient perspective?
![Page 18: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/18.jpg)
Understanding Local Provision of Care (Micro-systems)
Quality ImprovementPatterns of Care Analyses
Patient Safety
Major problems?
![Page 19: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/19.jpg)
![Page 20: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/20.jpg)
“The mere existence of evidence-based practices, however, does not always result in the use of such practices because of the complexities inherent in translating evidence into practice.” How do we better understand these factors?
![Page 21: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/21.jpg)
![Page 22: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/22.jpg)
Toolbox: Qualitative research – Observation, key informant interviews, Focus groups
![Page 23: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/23.jpg)
In-depth phone and in-person interviews were conducted between October 2006 and September 2007 with 86 participants (31 physicians). Active resistance to evidence-based practice change was pervasive. Successful efforts to overcome active resisters included benchmarking infection rates, identifying effective champions, and participating in collaborative efforts.
![Page 24: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/24.jpg)
![Page 25: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/25.jpg)
![Page 26: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/26.jpg)
Complexity in frontline systems
• Complex assembly of people, information, resources, equipment and procedures working toward a common goal
• System vulnerability reflects exposure to
events and factors that can make the system less safe or more prone to adverse events
![Page 27: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/27.jpg)
Toolbox: Human factors engineering, systems analysis
Human Factors Engineering: Design systems and environments for safer, more effective and more efficient use by humans
![Page 28: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/28.jpg)
![Page 29: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/29.jpg)
“There is a general consensus by intensivists and nonsurgical providers that surgeons hesitate to withdraw life-sustaining therapy on their operative patients despite a patient's or surrogate's request to do so.” How do you better understand this?
![Page 30: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/30.jpg)
![Page 31: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/31.jpg)
Healthcare Delivery System(Macro-system)
Policy EvaluationPayment Reform
Work Force
Major problems?
![Page 32: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/32.jpg)
![Page 33: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/33.jpg)
VS.
Problem: Patients getting treatment A and B are different
![Page 34: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/34.jpg)
Evidence supporting policy change
What we know: Outcomes in Medicare patients improved after the national coverage decision
What we don’t know:
Improvements due to secular trends or the policy itself?
![Page 35: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/35.jpg)
Change over time
![Page 36: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/36.jpg)
Toolbox: Health economics, econometrics, natural experiments, health policy
![Page 37: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/37.jpg)
Differences-in-differences technique
Isolates the impact of a policy change on outcomes above and beyond any changes to a control group
Policy implementation
After Before
Out
com
es
Time
Expected Improvement
A B
Impact of policy change
Control
Intervention
![Page 38: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/38.jpg)
Toolbox: Health economics, econometrics, natural experiments, health policy
Difference-in-difference methods – Econometric techniques for “subtracting out” effect of time Instrumental variables analysis – Econometric technique for isolating causal effects of interventions
![Page 39: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/39.jpg)
Evaluating Disease Management (Patient-Level)
Healthcare Delivery System (Macro-systems)
Understanding Local Provision of Care (Micro-systems)
Patient-Centered Outcomes Comparative Effectiveness
Data Synthesis
Quality Improvement Patterns of Care Analyses
Patient Safety Policy Evaluation Payment Reform
Work Force
Coverage Decisions Technology Adoption
Cost Effectiveness
Implementation Science
Quality Measurement
![Page 40: Understanding the Outcomes Research Toolboxweb2.facs.org/.../D1-S3-DimickPowerpoint.pdfUnderstanding the Outcomes Research Toolbox Justin B. Dimick, MD, MPH Henry King Ransom Professor](https://reader034.fdocuments.in/reader034/viewer/2022050501/5f93c2defdeaa34e0b77cef9/html5/thumbnails/40.jpg)
Major theme Research areas Research tools Resources
Evaluation of disease management (patient level)
• Comparative effectiveness
• Patient centered outcomes
• Data synthesis
• Randomized trials • Pragmatic trials • Advanced
observational study designs
Understanding local health care systems (micro-systems)
• Quality measurement and improvement
• Patterns of Care Analyses
• Patient Safety
• Qualitative research methods
• Human factors theory
• Process engineering
Understanding influence of healthcare delivery system (macro-systems)
• Policy Evaluation • Payment Reform • Surgical work force
• Large database analyses
• Econometric methods
• Health care policy