Tech and Risk Adjustment Using Data Analytics and Machine ...
47
Tech and Risk Adjustment: Using Data Analytics and Machine Learning to improve Risk Adjustment Management NPA Conference 10/25/2016
Transcript of Tech and Risk Adjustment Using Data Analytics and Machine ...
Tech and Risk Adjustment: Using Data Analytics and
Machine Learning to improve Risk Adjustment Management
NPA Conference 10/25/2016
On Lok Risk Adjustment Overview
▪Challenges▪Problems Attempting to Solve▪Tech Solutions
2
Agenda
•Sam – Technology and Reports•Kat – Reports Workflow•Travis – Machine Learning •Kat – Coder Workflows, Findings, and Provider Education
3
Tech Solutions Focus Areas
2 Main Focus Areas
•Free Text in Provider Documentation – Machine Learning and Chart Review Tool
•Missed or Incomplete RAPS Documentation in EHR - Operational Reports
Tech Solutions:Tech Solution
4
Machine Learning
Chart Review Tool
Operational Reports
Outcomes
Corrected Charts
Machine Learning
Chart Review
Operational Reports
Comprehensive HCC submissions
Improved Patient Documentation
Provider Education
Machine Learning – [RALE]
Risk Adjustment Learning Engine System•Machine Learning Engine•Provider Queues•Addendum Creation•Monitoring and Metrics•Generates Dx for RAPS
6
Chart Reviews – [Doc. Coder]
Tool to Assist with Chart Reviews•Advanced EHR Searching•RALE Integration•Provider Queues•Addendum Creation•Generates Dx for RAPS•Streamlines workflow
7
Operational HCC Reports
Missed Reporting
Opportunities< 2 HCC’s
Missing HCC after Cap
Incomplete Documentation
Missed Submission Opportunity
Dropped HCCProblems
recorded but not Assessed
Missed coding opportunity
Machine Learning
Chart Reviews/Doc
Coder
Monthly RAPS Processing Cycle
9
HCC Reports/Analysis
Extract/Submit EHR, Claims,
RALE, and CR
Chart updates
Machine Learning, Chart
Reviews
Next Up
•Kat – Reports Workflow•Travis – Machine Learning •Kat – Workflows, Findings, and Provider Education
10
Reports Work Flow
•Missing E&M Code: automatically distributed weekly to coder, who reviews SOAP note and determines E&M code•Missing Master IM: automatically distributed weekly to provider, who completes SOAP template and generates note•>2 HCCs, Dropped HCC, and Missing HCC: distributed periodically to provider, who determines whether or not patient needs to be brought in for assessment
Machine Learning
Missed Reporting
Opportunities0 or 1 HCC
Missing HCC after Cap
Incomplete Documentation
Missed Submission Opportunity
Dropped HCCProblems
recorded but not Assessed
Missed coding opportunity
Machine Learning
Chart Reviews/Doc
Coder
Machine Learning
•NPA - On Lok Machine Learning
13
Machine Learning
Predicting Hierarchical Condition Categories
What is machine learning?
Making computers understand the meaning of observations
Categorized Observations Machine Learning
Algorithm Model
Where is Machine Learning?
● Your messaging app● The search bar● The Tesla● Your travel app● Your broker● And a billion other things
How does On Lok Apply Machine Learning?
● Observations = Chart Notes● Each chart note has a related diagnosis● Each diagnosis falls into an HCC● Collection of categorized chart notes = training set● Training set + machine learning = model● Model + new chart notes = prediction
On Lok example
HCC19 - Diabetes without complication
● Find chart notes w/diagnosis codes mapping to HCC19● This is the training set● The machine learning engine extracts the best features
from the documents. For HCC19 they could include: thirsty, hunger, losing weight, tired, etc
● The model is built and a subset of the training set is used to score the model’s accuracy
● We now have a model that we can run any new chart notes against
Making PredictionsWe have models, now what?
A new chart note is created
Run the note against the
model
Generate Predictions
Remove if HCC(s) already
assigned
Is this a high quality
prediction?
Add to the queue for the
provider
Provider queue review
Prediction is correct
Addendum added to chart
Results fed back into the
modelHCC submitted
Prediction Queue
Prediction Queue
Prediction Queue
Too good to be true?
Let’s look at the challenges with machine learning
Rocket Science
Data Dilemmas
We Still Need Humans
Models need attention
Rocket Science
There are a lot of moving parts, big concepts and math
Rocket Science
We use a collaboration of:
Data Scientists
Software Engineers
Database Administrators
Coders
Physicians
Rocket Science
Gather the data
Organize the data
Clean the data
Test the data
Build the models
Verify the models
Predict on new documents
Present the predictions
Create Addendums
Analyze the outcomes
We have applications to…
Data Dilemmas
QuantityToo few examples
QualityBad examples
QuantitySiamese Burmese
So what is this?
Quantity
Chart Notes
● On Lok has a small number of patients which generates a relatively small number of documents
● Machine learning is learn by example, a shallow training set will yield shallow learning
● Patient data is protected and therefore hard to come by● We are always looking for ways to find more categorized
data.
QualitySiamese Burmese
So what is this?
Quality
Our data wasn’t gathered with machine learning in mind
● Chart notes sometimes contain nothing that indicates the diagnosis attached to them
● When reviewing the predictions the predicted HCC category may match the participant but not the document
We Still Need Humans
● Because predictions are never 100% accurate, we need a human to verify the results (the queue mentioned earlier)
● If the predictions are < 50% accuracy it means there are more misses than hits. That can get very frustrating for a busy physician
● Frustrated physicians lead to lower participation and less accurate results
Models Need Attention
● Unless you have millions of really high quality documents, you will have to frequently rebuild and tune your models.
● Every time you get a pile of high quality results you will want to add them back into your model
● Models can take a long time to build● We frequently Check our models against actual results
What’s Next?
● Integration wherever documents are used○ EHR integration○ etc.
● Faster on-demand predictions○ Composite Model○ GPU processing
● Accuracy Improvements○ Composite Model○ Expanded Datasets○ Integration of Google’s NLP tools
RALE Workflow
RALE predictions run
Providers validate
predictions
Dx selected and addendum
created
RAPS file created
Dx and addendum added
to HER
Matches and No Matches fed back
into RALE
Missed Reporting
Opportunities0 or 1 HCC
Missing HCC after Cap
Incomplete Documentation
Missed Submission Opportunity
Dropped HCCProblems
recorded but not Assessed
Missed coding opportunity
Machine Learning
Chart Reviews/Doc
Coder
Doc Coder
Document Coder Workflow - K
Recommendations entered into Doc
Coder.Providers validate
suggestionsSelect dx and
addendum created.
RAPS file created.Dx and addendum added to EHR.
Matches and No Matches are fed back
into RALE.
Doc Coder – Results - K
38
Findings - Unique HCC’s Submitted for 2014 DOS
•As of 1/29/2016•Reports: 229 (4.9%)•Machine Learning: 336 (7.4%)•Doc Coder: 220 (4.8%)•Improvement in HCC risk scores 2014/2015 Dates of Service:• Base: +21% (provider education)• Tools: +17% (Rpts, Mach Lrn, Doc Coder)
39
Top Three Machine Learning Findings
1. Vascular Disease (21%)•Specifically, atherosclerosis of the aorta
2. Major Depressive, Bipolar, and Paranoid Disorders (18%)
•Specifically, major depression3. Dementia without Complication
(13%)
40
Top Three Doc Coder Findings
1. Major Depressive, Bipolar, and Paranoid Disorders (18%)
•Specifically, Major Depression2. Polyneuropathy (14%)3. Vascular Disease (12%)
41
Education of Providers(Kat)
•Diagnostic sources• Review all labs, specialist referrals, etc. on re-eval
• Example: atherosclerosis of aorta on chest x-rays
•Diagnostic guidelines• Review medical journals for diagnostic requirements
• Example: depression vs. major depressive disorder
•Coding Guidelines• Review coding resources for coding requirements
• Example: dementia with vs. without complications
42
RAPS Processing Overview
44
Run Tools and Reports
Validate and update Charts
Extract Claims, EHR, Machine Learning, Doc
Coder
Submit to RAPS
Process Errors
Missed documentationMissed face to face encounters
Add E&M code and Dx
Extract face to face encounters
RAPS return filesstored in EDW
Run monthly and during sweeps
Tools/Technology (Sam)
•Analytics•Reports•Machine Learning•Productivity Tools
•Doc Coder
45
Miss HCC
0 or 1 HCC
Missing HCC after Cap
Incomplete Documentation
Missed Submission Opportunity
Dropped HCCProblems
recorded but not Assessed
Missed coding opportunity
RALE
Chart Reviews/Doc
Coder
HCC’s
HCC Count103 2011108 3791111 4452122 67134 462136 1704138 5539139 4623169 8218 956228 35548 59451 411752 979255 24758 330575 60578 175579 81484 823388 18519 45996 5377 Total 69,797
46
About Doc Coder – PLACE Holder
•Assists with Manual Chart Reviews•Advanced Search Functionality•Knowledge of RALE Findings •Builds PCP Review Queue •Automated Generation of Addenda•Generates DX Code Files for RAPS
47
