Pu ZPIC Zone 4 Decision Tree Modeling - Sitemason ZPIC Zone 4 Decision Tree Modeling.pdfDecision...

1

A wholly owned subsidiary of Quality Health Strategies

Zone Program Integrity Contractor Zone 4Decision Tree Modeling

Holly Pu, M.S.

Chief Statistician

October 14, 2009


Data Project

Home Health

– Overview

– Fraud Indicators

– Decision Trees


Overview

2


Home Health Rules and Regulations

• 42 CFR (Code of Federal Regulations)

– Sections 409.40‐409.50 – Home Health Services

– Section 424.22 – Requirements for Home Health Services

– Section 484 – Home Health Conditions of Participation


Home Health Rules and Regulations

•• Publications 100Publications 100‐‐XXXXPublication 100‐01, Medicare General Information, Eligibility and Entitlement Manual , Chapter 1, Section 10.2, Home Health Services and Chapter 4, Section 30, Physician Certification

Publication 100‐02, Medicare Benefit Policy Manual, Chapter 7, Home Health Services

Publication 100‐04, Medicare Claims Processing Manual, Chapter 10, Home Health Agency Billing

Publication 100‐08, Program Integrity Manual (PIM)


Home Health Eligibility

• To qualify for Medicare coverage of home health services, the patient must meet each of the following:

– Services billed from a Medicare participating Home Health Agency

– Be confined to the home (homebound)

– Be under the care of a physician

– Be in need of skilled services

– Be under a Plan of Care

3


State County Summary

12,11653,140$29,497,595SAN JUAN NM

15,007125,816$38,260,750DONA ANA NM

24,8791511,774$63,050,653BERNALILLO NM

11,106126,727$29,105,530EL PASO CO

25,3661615,896$71,196,049ARAPAHOE CO

28,6432617,413$76,212,907DENVER CO

39,21656,408$81,676,418PUSHMATAHA OK

68,7862822,482$167,193,549TULSA OK

102,4665431,000$256,388,647OKLAHOMA OK

150,30211749,424$444,553,066BEXAR TX

169,15410350,356$483,479,760TARRANT TX

329,49111654,175$866,242,700HIDALGO TX

329,05236775,354$1,011,769,175DALLAS TX

376,68346293,954$1,073,334,215HARRIS TX

Number of Claims

Number of Home Health Agencies

Number of PatientsTotal PaymentProvider County

Provider State



Growth Trend

Trend by Payment Amount

$0

$50,000,000

$100,000,000

$150,000,000

$200,000,000

$250,000,000

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep Oct

Nov

Dec Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep Oct

Nov

Dec Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep Oct

Nov

Dec

2006 2007 2008

CO Payment NM Payment OK Payment TX Payment

4


Fraud Indicators


Extensive Home Care Episodes

• Background: Home health care is usually temporary. The goal of home health care is rehabilitation. If a home health agency has a high percentage of beneficiaries receiving long term home care services, it may indicate fraud/abuse.

• Time Frame: April, 2006 – March 31, 2009

• Findings: In three years, Medicare paid around $4.2 billion for beneficiaries with five or more consecutive home care episodes. Out of the $4.2 billion, over $1.1 billion were paid for beneficiaries with twelve or more consecutive home care episodes. An initial threshold of 40 percent was established by the home health subject matter expert to flag aberrant providers.


Consecutive Home Care EpisodesSummary by State

$4,163,283,216Grand Total

$3,549,824,046Subtotal

28,029$985,801,179More than 12 Episodes

1,565157,193$2,564,022,867(More than 5 Episodes, Less than 12 Episodes)

TX

$541,286,690Subtotal

5,796$168,912,079More than 12 Episodes

20523,449$372,374,611(More than 5 Episodes, Less than 12 Episodes)

OK

$50,855,557Subtotal

408$11,890,298More than 12 Episodes


NM

$21,316,923Subtotal

168$4,579,632More than 12 Episodes


CO

Number of Home Health AgenciesNumber of PatientsPaymentNumber of Home Health Episodes

Provider State

5


Outlier Payment Claim

• Background: Home Health outlier payment is identified by the value code 17. Additional PPS reimbursement is based on the number of visits, often in the $10,000 range. PPS assumes the cost of care exceeds the threshold dollar amount resulting in an outlier payment.

• On July 30, 2009 CMS proposes to cap outlier payments at 10 percent per agency and target total aggregate outlier payments at 2.5 percent of total HH PPS payments for calendar year 2010. An initial threshold of 50 percent was established by the home health subject matter expert to flag aberrant providers.


Home Health Outlier – Summary by County

0.060.080.046,7382,014$13,865,001245159$854,118EDDYNM

0.040.040.0315,0075,816$38,260,750393256$1,405,791DONA ANANM

0.030.020.0225,35515,887$71,182,890403260$2,039,689ARAPAHOECO

0.060.030.0328,44317,272$75,635,586875465$4,286,026DENVERCO

0.070.030.0368,75022,447$167,094,6122,007742$12,322,113TULSAOK

0.110.050.04101,96830,804$255,233,5564,5271,619$28,603,974OKLAHOMAOK

0.070.030.03278,77578,497$798,415,3257,6932,631$55,210,502HARRISTX

0.160.050.06168,26750,079$481,189,36210,7062,749$76,372,653TARRANTTX

0.260.080.12149,82749,208$443,424,19518,5854,052$115,709,588BEXARTX

0.210.120.10329,03154,075$865,246,00133,3116,395$185,583,072HIDALGOTX

0.250.080.10322,71274,472$996,815,92931,6725,945$248,218,141DALLASTX

Ratio Payment (Outlier Payment / Total Payment)

Ratio Pats (Outlier Patients / Total Patients)

Ratio Claims (Outlier Claims / Total Claims)

Total Claims

Total PatientsTotal Payment

Outlier Claims

Outlier PatientsOutlier Payment

Provider County

Provider State


Home Health Outlier – Summary by State

• Findings: There were over 100 home health agencies that had 50 percent or more outlier payment claims. Out of these providers the majority concentrates in the State of Texas. The total outlier payment made in ZPIC Zone 4 states was over $270 million in a three‐year time period.

$429,589,221$274,367,004127Grand Total

$410,093,209$263,880,584117TX

$17,784,424$9,478,0294OK

$1,711,589$1,008,3916CO

Total Payment (for the Home

Health Agencies with 50% or more

Outlier Payment Claims)Outlier Payment

Number of Home Health Agencies

(with 50% or more Outlier

Payment Claims)Provider State

6


Single Referring Part B Physician

• Background: This indicator is to look for home health referring providers who referred the majority of the patients to home health agencies. An initial threshold of 40 percent of claims was established to flag aberrant providers.

• Time Frame: April 1, 2006 – March 31, 2009

• Findings: Thirty‐one home health agencies had 40 percent or more of their patients referred by an individual Part B physician. The home health payment referred by these physicians was over $28 million. Some of these agencies shared one common Part B Physician who is currently under investigation.


HIPPS Code Indicators

• 1st position ‐ prior to 2008 always an H, effective 1/1/2008 numbers 1‐5, dependent on episode and therapy services

• 2nd position ‐ Clinical Severity Domain• 3rd position ‐ Functional Status Domain• 4th position ‐ Service Utilization Domain• 5th position ‐ prior to 2008 data validity flag, effective

1/1/2008 alpha/numeric related to supplies


Power of a HIPPS Code ‐ Therapy

RangeRange--$1,800$1,800--$2,200$2,200$7,023.96$7,023.965BGKW 5BGKW (20+ therapy) (20+ therapy)

$2,979.11$2,979.11$5,220.86$5,220.864BGKW 4BGKW (14(14--19 therapy)19 therapy)

$2,241.75$2,241.753BGKW 3BGKW (0(0--13 therapy)13 therapy)

$2,626.32$2,626.32$4,813.12$4,813.122BGKW 2BGKW (14(14--19 therapy)19 therapy)

$2,186.80$2,186.801BGKW 1BGKW (0(0--13 therapy)13 therapy)

Increase in $$Increase in $$PPS ReimbursementPPS ReimbursementHIPPS CodeHIPPS Code

7


HIPPS Code First Position ‐ Therapy Visits

• Background: Medicare made changes to the Home Health Prospective Payment System (HHPPS) effective January 1, 2008. The first position of a HIPPS code with a value 5 signifies 20 or more therapy visits. An initial threshold of 40 percent of claims was established to flag aberrant providers.

• Time Frame: February 1, 2008 – January 31, 2009 (1 Year)

• Findings: 4 home health agencies were selected for further investigationin that they had a percentage as high as 87 percent of billing the HIPPS code with a value 5 in the first position. The total payment for this HIPPS code billed by these 4 providers was over $4.5 million in a one‐year time period.


HIPPS Code Second Position ‐ Clinical Severity

• Background: The second position of a HIPPS code with a value C signifies the most severe clinical condition. An initial threshold of 40 percent of claimswas established to flag aberrant providers.



HIPPS Code Third Position ‐ Functional Status

• Background: The third position of a HIPPS code with a value H signifies the lowest level of functional status. An initial threshold of 40 percent of claimswas established to flag aberrant providers.


8


Static Diagnosis Code

• Background: This indicator is to look for home health agencies billing the same diagnosis codes for the majority of their beneficiaries. An initial threshold of 50 percent of claims was established to flag aberrant providers.

• Time Frame: April 1, 2006 – March 31, 2009


Exact Five Visits per Claim

• Background: Medicare makes a low‐utilization payment adjustment (LUPA), if at the end of an episode there have been four or fewer services. The home health agency will be compensated based on a per visit amount, which is much less than an episode rate.

• This indicator looks for home health agencies with a high percentage of beneficiaries with exact five visits per claim. An initial threshold of 40 percent of claims was established to flag aberrant providers.


Decision Tree

9


Predictive Modeling

• Predictive models depend on a property known statistically as stationarity, meaning that its statistical properties do not change over time.

• Training Data: Predictive Modeling starts with a training data set. The training data set consists of cases (also known as observations, examples, instances, or records). Associated with each case is a vector of input variables (also known as predictors, explanatory variables) and a target variable (also known as a response, outcome, or dependent variable).

• Score Data: Predictive Modeling ends with a score data set. The score data set has the same structure as the training data set, but lacks a target variable.

• Predictions: The bridge between the training and score data is the predictions generated by the analysis. The predictions represent the best guess for the unknown target variable score data, based on the known input variables and associations between the input variables and target variable learned from the training data.


Model Essentials

• Predict New Cases: the simplest type of prediction is the decision. Decisions usually are associated with some kind of action (such as classifying a home health agency as potentially fraudulent or not fraudulent). For this reason, decisions are also known as classifications.

• Select Useful Inputs: to select an independent set of inputs that are correlated with the target.

• Optimize Complexity: Underfitting versus. Overfitting


Data Splitting

• For honest assessment of model performance, data splitting is performed. A portion is used for fitting the model, that is, the training data set. The remaining data is separated for empirical validation.

• The validation data set is used for monitoring and tuning the model to improve its generalization. The tuning process usually involves selecting among models of different types and complexities.

10


Decision Trees

• Decision Trees addresses each of the modeling essentials described earlier. Cases (home health agencies) are scored using prediction rules. A split search algorithm facilitates input selection. Model complexity is addressed by pruning.

• Decision Tree Prediction Rules: The rules are arranged hierarchically in a tree‐like structure with nodes connected by lines. The nodes represent decision rules, and the lines order the rules.


Decision Tree Split Search• Split Search starts by selecting an input for partitioning the available training data.

For a selected input and fixed split point, two groups (branches) are generated. A Pearson Chi‐Square statistic is used to test if the proportion of zeroes and ones in the left branch is significantly different than the proportion in the right branch. A large difference in outcome proportions indicates a good split.

• The statistic is converted to a p‐value. For large data sets, these p‐values can be very close to zero. For this reason, the quality of a split is reported by logworth = ‐log(chi‐squared p‐value). SAS Enterprise Miner uses a logworth of approximately 0.7.

• The process repeats until there are no more allowed splits whose logworth exceeds the thresholds. The resulting partition of the input space is known as the maximal tree. It is likely that the maximal tree will fail to generalize well on an independent set of validation data.


Decision Tree Split Search

11


Pruning a Decision Tree

• To avoid potential overfitting, many predictive modeling procedures offer some mechanism for adjusting model complexity. For decision trees, this process is known as pruning. The general idea of pruning is to select the simplest model with the best validation performance.

• Misclassificationmeasures the fraction of cases where the decision does not match the actual target value. Decisions require low misclassification.


Pruning a Decision Tree


Conclusion• In three years, Medicare paid around $4.2 billion for beneficiaries with five or

more consecutive home care episodes. Out of the $4.2 billion, over $1.1 billion were paid for beneficiaries with twelve or more consecutive home care episodes.

• There were over 100 home health agencies that had 50 percent or more outlier payment claims. Out of these providers the majority concentrates in the State of Texas. The total outlier payment made in ZPIC Zone 4 states was over $270 million in a three‐year time period.

• Based on the fraud indicators and decision tree rules, an initial selection of 24 home health agencies currently are under investigation. The total Medicare payment at risk is over $365 million.

• Investigative Findings: These providers were found to bill for a large percentage of their patients who were either not home bound, not in need of a skilled level of care, or have been educated to self‐administer insulin injections for an extended period of years. Due to the high volume of fraud found in our states, a separate task order was initiated for this effort.

12


References

• Applied Analytics Using SAS Enterprise Miner 5

• Code of Federal Regulations

• CMS Manual Publication 100‐01, 100‐02, 100‐03, 100‐04


Contact Information

Holly Pu

Chief StatisticianHealth Integrity, LLC9240 Centreville RoadEaston, MD 21601email: [email protected]: 410‐770‐3058Fax: 410‐819‐8698

Pu ZPIC Zone 4 Decision Tree Modeling - Sitemason ZPIC Zone 4 Decision Tree Modeling.pdfDecision...

Documents

Transcript of Pu ZPIC Zone 4 Decision Tree Modeling - Sitemason ZPIC Zone 4 Decision Tree Modeling.pdfDecision...