There Is A 90% Probability That Your Son Is Pregnant: Predicting the Future of Predictive Analytics...
-
Upload
health-catalyst -
Category
Healthcare
-
view
2.702 -
download
1
Transcript of There Is A 90% Probability That Your Son Is Pregnant: Predicting the Future of Predictive Analytics...
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
© 2014 Health Catalystwww.healthcatalyst.comProprietary and ConfidentialFollow Us on Twitter #TimeforAnalytics
Predicting The Future Of Predictive Analytics In Healthcare
There’s A 90% Chance Your Son Is Pregnant
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 2
Presenter and Contact Information
Dale SandersSenior Vice President, Strategy, Health [email protected]@drsanderswww.linkedin.com/in/dalersanders/
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Acknowledgements
David Crockett, PhD, Health Catalyst
Eric Siegel, PhD, Columbia University
Ron Gault, Aerospace Corporation, Northrup-Grumman, TRW
Wikipedia
3
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
The Goal Today
I hope you leave this webinar with…
Informed Expectations and Opinions: Be generally aware of the realistic possibilities for predictive analytics in healthcare, over the next few years
The Right Questions: To be conversant in the concepts of predictive analytics and be able to ask reasonably well-informed questions of your analytics teams, especially vendors, during the strategic process of developing your organization’s predictive analytics strategy
4
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
5
Agenda5
Basic Concepts, Fundamental Assertions
Predictive Analytics Outside Healthcare
Predictive Analytics Inside Healthcare
Key Questions To Ask Vendors And Your Analytics Teams
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 6
Sampling of My Background In Predictive Analytics
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
7
7
Gartner 2014 Hype Cycle for Emerging Technology
Predictive Analytics in Healthcare, according to Dale Sanders
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
“Beyond math, there are no facts; only interpretations.”
- Friedrich Nietzsche
8
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Challenge of Predicting Anything Human
9
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 10
What Should We Expect In Healthcare?Machines are predictable; humans aren’t
“People are influenced by their environment in innumerable ways. Trying to understand what people will do next, assumes that all the influential variables can be known and measured accurately. People's environments change even more quickly than they themselves do. Everything from the weather to their relationship with their mother can change the way people think and act. All of those variables are unpredictable. How they will impact a person is even less predictable. If put in the exact same situation tomorrow, they may make a completely different decision. This means that a statistical prediction is only valid in sterile laboratory conditions, which suddenly isn't as useful as it seemed before.”
Gary King, Harvard University and the Director of the Institute for Quantitative Social Science
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Healthcare Analytics Adoption Model
Level 8
Level 7
Level 6
Level 5
Level 4
Level 3
Level 2
Level 1
Level 0
Personalized Medicine& Prescriptive Analytics
Clinical Risk Intervention& Predictive Analytics
Population Health Management& Suggestive Analytics
Waste & Care Variability Reduction
Automated External Reporting
Automated Internal Reporting
Standardized Vocabulary& Patient Registries
Enterprise Data Warehouse
Fragmented Point Solutions
Tailoring patient care based on population outcomes and genomic data. Fee-for-quality rewards health maintenance.
Organizational processes for intervention are supported with predictive risk models. Fee-for-quality includes fixed per capita payment.
Tailoring patient care based on population metrics. Fee-for-quality includes bundled per case payment.
Reducing variability in care processes. Focusing on internal optimization and waste reduction.
Efficient, consistent production of reports & adaptability to changing requirements.
Efficient, consistent production of reports & widespread availability in the organization.
Relating and organizing the core data content.
Collecting and integrating the core data content.
Inefficient, inconsistent versions of the truth. Cumbersome internal and external reporting.
© Sanders, Protti, Burton, 2013
11
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Concepts & Principles of Predictive Analytics
12
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 13
Semantics, SsschmanticsPredictive Analytics and Predictive Models: These terms have their origins in statisticians; e.g., understanding real-world phenomena such as healthcare, retail sales, customer relationship management, voting preferences, etc.
Machine Learning Algorithms: This term has its origins in computer scientists; e.g., natural language processing, speech recognition, image recognition, adaptive control systems in manufacturing, robots, satellites, automobiles and aircraft, etc.
As it turns out, the latter can be applied to the former, so the two schools of thought are now generally interchangeable. Don’t let vendors fool you into thinking that “machine learning” is more sophisticated or better than
predictive modeling.
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
14
14
For Now, Just Know The TermsAnd Know Where To Go For Details
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
For Now, Just Know The TermsAnd Know Where To Go For Details
MachineLearningMastery.com
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
The Basic Process of Predictive Analytics
16
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
17
A Big & Common Mistake: Over Fitting
17You train the model to be very specific on a given data set, but the model cannot adapt to a new, unknown data set
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Specificity vs. Sensitivity: Trading One For Another
Specificity:
The true negative rate. For example, the percentage of diabetic patients identified who will not have a myocardial infarction
Sensitivity:
The true positive rate. For example, the percentage of diabetic patients that will have a myocardial infarction
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 19
Receiver Operating Characteristic (ROC) Plot
• Tuning radar receivers in WWII• Maximum radar receiver sensitivity led
to many false positives… too many alarms
• Lower radar receiver sensitivity led to many false negatives… missed threats
• Same challenge in airport security screening systems and spam filters
• Concept has been applied heavily in diagnostic medicine
• True Positive Rate vs. False Positive Rate
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Data Volume vs. Predictive Model
“But invariably, simple models and a lot of data trump more elaborate models based on less data.”
“The Unreasonable Effectiveness of Data”, March 2009, IEEE Computer Society; Alon Halevy, Peter Norvig, and Fernando Pereira, Google
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
The Human Data Ecosystem
21
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
We Are Not “Big Data” in Healthcare Yet
22
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Predictive Precision vs. Data Content
23
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Thank you for the graphs, PreSonus
Healthcare and patients are continuous flow, analog process and beings
But, if we sample that analog process enough, we can approximately recreate it with digital data
24
Remember Your Calculus Digital Sampling Theory?
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
We are asking physicians and nurses to act as our “digital samplers”… and that’s not going to work
25
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Predictive Analytics Outside HealthcarePredictive Analytics Outside Healthcare
26
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 27
“Mr. Sanders, while your 9-year tenure as an inmate has been stellar, our analytics models predict that you are 87% likely to become a repeat offender if you are granted parole. Therefore, your parole is denied.”
- 2014, 80% of parole boards now use predictive analytics for case management*
* The Economist, “Big data can help states decide whom to release from prison” April 19, 2014
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Thank you Sonja Star, New York Times
“Evidence Based” Sentencing20 states use predictive analytics risk assessments to inform criminal sentencing.
28
“Evidence Based” Sentencing
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Recidivism Risk Assessment: Level of Service/Case Management Inventory (LS/CMI)*
29
15 different scales feed the PA algorithm
1. Criminal history
2. Education/employment
3. Family/marital
4. Leisure/recreation
5. Companions
6. Alcohol/drug problems
7. Antisocial patterns
8. Pro-criminal attitude orientation
9. Barriers to release
10. Case management plan
11. Progress record
12. Discharge summary
13. Specific risk/needs factors
14. Prison experience - institutional factors
15. Special responsivity consideration
42.2% of high-risk offenders recidivate within 3 years *Nov. 2012, Hennepin County, Minn. Department of Community Corrections and Rehabilitation
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
30
“Since the publishing of Lewis' book, there has been an explosion in the use of data analytics to identify patterns of human behavior and experience and bring new insights to fields of nearly every kind.”
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 31
eHarmony Predictions“Heart” of the system: Compatibility Match Processor (CMP)
• 320 profiling questions/attributes per user• 29 dimensions of compatibility• ~75TB• 20M users• 3B potential matches daily• 60M+ queries per day, 250 attributes
Thank you, Thod Nugyen, eHarmony CTO
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 32Thank you, Ryan Barker, Principal Software Engineering – Matching, eHarmony
29 Dimensions of Compatibility
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Predictive Analytics Inside Healthcare
33
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 34
What Are We Trying to Predict?
Common applications being marketed today
• Identifying preventable re-admissions: COPD, MI/CHF, Pneumonia, et al
• Sepsis• Risk of decubitus ulcers• LOS predictions in hospital and ICU• Cost-per-patient per inpatient stay• Cost-per-patient per year by disease and comorbidity• Risk of ICU mortality• Risk of ICU admission• Appropriateness of C-section• Emerging: Genomic phenotyping
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 35
True Population Predictive Risk Management
Thank you, for the diagram, Robert Wood Johnson Foundation, 2014
Very Little ACO Influence
Very Little ACO Influence
>/=30% Waste*100% ACO Influence
*Congressional Budget Office, IOM, “Best Care at Lower Cost”, 2013
True Population Health Management
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 36
Not all patients can functionally participate in a protocol
At Northwestern (2007-2009), we found that 30% of patients fell into one or more of these categories:
• Cognitive inability• Economic inability• Physical inability• Geographic inability• Religious beliefs• Contraindications to the protocol• Voluntarily non-compliant
Socioeconomic Data Matters
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
37
The key to predictive analytics in the future of health care will be the ability to answer this two-part question:
What’s the probability of influencing this patient’s behavior towards our desired outcome and how much effort (cost) will be required for that influence?
Return on Engagement (ROE)
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
38
Return on Engagement (ROE)
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 39
Socioeconomic Data Matters
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
40
40
Development Partner
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Flight Path “Outcomes”
41
Examples
Diabetes Cohort
Good Flight Path
Poor Flight Path
1. A1c < 72. LDL < 1003. BP < 130/80
1. A1c > 7 2. LDL > 100 3. BP > 130/80
$ COST Per Member Per Year (Charges)
For > 1 year of encounters
(~5 yrs and 26k patients)
These aren’t really outcomes… they are proxies for outcomes
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
True Outcomes
42
Good Flight Path
Poor Flight Path
Absence of:Cardiovascular disease (angina, MI, stroke)Nephropathy/End stage renalDiabetic retinopathyGlaucomaCataractsLower extremity tissue narcosis, foot ulcersPeripheral neuropathyDiabetic ketoacidosisDiabetic preeclampsiaGI complications (nausea, constipation)Erectile dysfunctionPresence of:Cardiovascular disease (angina, MI, stroke)Nephropathy/End stage renalDiabetic retinopathyGlaucomaCataractsLower extremity tissue narcosis, foot ulcersPeripheral neuropathyDiabetic ketoacidosisDiabetic preeclampsiaGI complications (nausea, constipation)Erectile dysfunction
Diabetes Cohort
(~5 yrs and 26k patients)
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
43
Two Layers of Predictive FunctionRisk scores Simulation
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
44
Microsoft Azure: Cloud-Based Algorithms
44
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Allina Health Readmissions Model*Variables Considered
*- Thank you, Jonathan Haupt
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Allina Compared To Other ModelsMultiple logistic regression
5.2% of discharged patients in high risk category
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Allina’s Intervention To Reduce RiskTransition of Care “Conferences”
• Patients, families, care givers
• 15% reduction in readmissions
• 100+ APR-DRGs affected
• More patients utilizing post-acute care‒ Skilled Nursing Facility
‒ Home Health
‒ TCU
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Antibiotic Protocol
Dosage Route Interval Predicted Efficacy
Average Cost/Patient
Option 1 500mg IV Q12 98% $7,256
Option 2 300mg IV Q24 96% $1,236
Option 3 40mg IV Q6 90% $1,759
• Predictive and prescriptive (suggestive) analytics in the same user interface• The efficacy and costs of antibiotic protocols for inpatients
Thank you, Dave Claussen, Scott Evans, et al, Intermountain Healthcare
48
The Antibiotic Assistant
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
The Antibiotic Assistant Impact• Complications declined 50%
• Avg. number of doses declined from 19 to 5.3
• The replicable and bigger story‒ Antibiotic cost per treated patient: $123 to $52‒ By simply displaying the cost to physicians
49
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Wrapping Up
50
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Key Questions To AskOf Vendors and Your Analytics Teams
51
1. What is your formal training, education, and practical
experience in this field?
2. What are the input variables to the model?
3. What model and/or algorithms are you using and why?
4. How are you going to train the model?
5. Are you using our data or other organizations’ data for
training? Why?
6. If you are using other organizations’ data, how are you going
to customize the model to our specific data environment?
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics 52
1. Action matters: What is the return in investment for intervention? Are we prepared to invest more... or say “no”… to patients who score low on predicted engagement?
2. Human unpredictability: The mathematical models of human behavior are relatively immature.
3. Socio-economics: Can today’s healthcare ecosystem expand to make a difference?
4. Missing data: Without patient outcomes, the PA models are open loop.
5. Social controversy: How much do we want to know about the future of our health, especially when the predictive models are uncertain?
6. Wisdom of crowds: Suggestive analytics from “wise crowds” might be easier and more reliable than predictive analytics, until our data content improves
Closing Thoughts and Questions
© 2014 Health Catalystwww.healthcatalyst.comFollow Us on Twitter #TimeforAnalytics
Q & A
53
• Submitted prior to the webinar
• Submitted through the webinar chat box
Thank YouFor questions and follow-up, please contact me• [email protected]• @drsanders
Upcoming Educational Opportunities
An Overview of the Healthcare Analytics MarketDate: January 21, 2015, 1-2pm, ESTHost: Jim Adams, Executive Director, The Advisory Board
A Pioneer ACO Case Study: Quality Improvement in HealthcareDate: January 28, 2015, 1-2pm, ESTHosts:Robert Sawicki, MD, Senior Vice President of Supportive Care, OSF HealthCare
Roopa Foulger, Executive Director Data Delivery, OSF HealthCare
Linda Fehr, RN, Division Director of Supportive Care, OSF HealthCare