Predictive Modeling Project

Stephen P. D’ArcyProfessor of Finance

University of Illinois at Urbana-Champaign

ORMIR PresentationOctober 26, 2005

Motivation - To Advance the Science of Predictive Modeling by:

• Applying predictive modeling to a key aspect of insurance operations

• Sharing the results of this research fully so that other researchers can replicate the results and improve the process

• Educating practitioners about the value of predictive modeling

• Opening up the “black box” approach of data mining that has generally been applied

Predictive Modeling in Insurance• Massive amounts of data available

– Accuracy varies– Much of it is ignored in rating or claims handling

• Innovators– Use of credit scoring in rating– Predictive modeling applications

• Underwriting• Claims handling• Fraud investigation

• Studies treated as proprietary and not shared or published

Project Details• Jointly funded by the National Center for Supercomputing

Applications (NCSA) and ORMIR

• Data set:– Detail Claim Database created by the Automobile Insurers Bureau

of Massachusetts

• Predictive modeling tool:– Data-to-Knowledge (D2K) program of NCSA

• Results:– Papers– Presentations

Steps in Predictive Modeling1. Decide question to be investigated2. Access data3. Understand your data4. Preliminary data mining analysis

• Decision trees• Generalized linear regression

5. Evaluate results and investigate problems6. Additional data mining analysis

• Trees and regression• Neural networks• Other techniques

7. Apply results to insurance operations8. Evaluate impact of change

Detail Claim Database (DCD)

• Created by the Automobile Insurers Bureau (AIB) of Massachusetts;

• Primary objectives:– Supporting company claim negotiation and claim denial

activities– Assisting the Board of Registration– Responding to the Division of Insurance and to the

Legislature– Assisting the Insurance Fraud Bureau of Massachusetts in

detecting possible fraud rings

• Accessible for all member companies of the AIB and selected researchers

DCD Observations and Variables• 491,591 Claim Observations (1/1/94 and subsequent)• 95 Variables from 5 Categories:

– Policy Information– Claim Information

• Coverage • Total amount paid• Accident date • Type of injury• Report date • Type of treatment

– Outpatient Medical Provider Information (up to 2 providers)• Provider type (MD, Chiropractor, Physical Therapist, Hospital, Other)• Amount billed and PIP/MED amount paid

– Attorney Information– Claim Handling Information

• Type of investigation, if any

Types of Investigations• Independent Medical Examination (IME)

– 66,876 Requests (16.72%)– Average Savings $348.71– Favorable Outcomes (60%)

• Medical Audit (MA)– 44,099 Requests (11.02%)– Average Savings $367.08– Favorable Outcomes (67%)

• Special Investigation (SI)– 16,668 Requests (4.17%)– Average Savings $1805.39– Favorable Outcomes (46%)

Problem – Average Savings values are based on aformula and may not reflect actual savings

Steps to Avoid Problems with Recorded Savings Value

1. Use Favorable Outcome as dependent variable

2. Generate value for expected payment • Stepwise linear regression (33 steps)• Based on claims not investigated• Apply to IME Requested claims• Compare expected payment to actual payment

Result of IME Mean (Expected – Actual)

No change recommended -562

Favorable result 18

Regression ResultsDependent Variable = Tot_PaidCoefficient Variable

MP1_TYPE MP2_TYPE PRIMTYPE 1894.004 InterceptCH 690.6566 CH 442.1823 CH -627.62 1 -173.80 Pol_Type=P + CO 668.4522 CO 414.3684 CO -536.863 2 -175.70 Em_Treat=B + MD 0 MD 0 MD -287.647 3 -670.98 Em_Treat=N + MI 0 MI 456.9774 MI 0 4 492.22 Health_I=N +

MO 0 MO 0 MO 0 5 -34.33 Health_I=U + N1 -971.328 N1 -392.816 N1 0 6 3020.23 Inj_Type=MJ + N2 -185.653 N2 0 N2 0 7 604.00 Inj_Type=SE + NO 0 NO -596.941 NO -646.206 8 690.66 MP1_TYPE=CH + PO 682.4842 PO 458.7032 PO -375.556 9 668.45 MP1_TYPE=CO + PT 644.808 PT 392.9716 PT -497.701 10 -971.33 MP1_TYPE=N1 +

11 -185.65 MP1_TYPE=N2 + 12 682.48 MP1_TYPE=PO +

Inj_Type Health_I INJ_GRP 13 644.81 MP1_TYPE=PT + MJ 3020.227 N 492.2223 1 -259.887 14 442.18 MP2_TYPE=CH +

MM 0 U -34.3341 2 0 15 414.37 MP2_TYPE=CO + SE 603.998 Y 0 3 0 16 456.98 MP2_TYPE=MI + SS 0 0 4 0 17 -392.82 MP2_TYPE=N1 +

18 -596.94 MP2_TYPE=NO + 19 458.70 MP2_TYPE=PO + 20 392.97 MP2_TYPE=PT +

ACCMONTH Pol_Type Em_Treat 21 -259.89 INJ_GRP=01 + Q1 0 P -173.8 B -175.699 22 1323.34 ATT + Q2 0 C 0 N -670.978 23 -627.62 PRIMTYPE=CH + Q3 0 Y 0 24 -536.86 PRIMTYPE=CO + Q4 49.58459 25 -287.65 PRIMTYPE=MD +

26 -646.21 PRIMTYPE=NO + 27 -375.56 PRIMTYPE=PO + 28 -497.70 PRIMTYPE=PT + 29 0.10 PRIMBILL + 30 49.58 ACCMONTH=Q4 + 31 92.12 TREATLAG + 32 -21.90 REP_LAGT + 33 10.05 CLMT_AGE +

Summary of Coefficients

Primary Medical Provider Types and Attorney Representation Frequency

Primary MP_Type v.s. Attorney

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

CH CO MD MI MO PO PT Total

CH

CO

MD

MI

MO

PO

PT

Total

Highest Attorney Representation by Individual Medical Provider

ATT=1 Total Claims ATT Freq PRIM_TYPE63 63 100.00% CO57 57 100.00% PO

212 214 99.07% PO83 84 98.81% MD78 79 98.73% MO61 62 98.39% CH

110 112 98.21% CO55 56 98.21% CH

155 158 98.10% CH102 104 98.08% MO152 155 98.06% CH

Injuries’ Seasonality TrendInjury 5 vs. Accident Month

0

5000

10000

15000

20000

25000

30000

35000

1 2 3 4 5 6 7 8 9 10 11 12

Accident Month

Total Injury

Injury 6 vs. Accident Month

0

500

1000

1500

2000

2500

3000

1 2 3 4 5 6 7 8 9 10 11 12

Accident Month

Total Injury

Injury 30 vs. Accident Month

0

200

400

600

800

1000

1200

1 2 3 4 5 6 7 8 9 10 11 12

Accident Month

Total Injury

Injury (exclud 5,6,30) vs. Accident Month

0

2000

4000

6000

8000

10000

12000

14000

1 2 3 4 5 6 7 8 9 10 11 12

Accident Month

Total Injury

Injury Type = SS

Primary Medical Provider = CH

Second Medical Provider

PIP Coverage Emergency Medical Treatment

Decision Tree Example

NY

YN

Decision Tree Approach for IMEs• Nodes and Favorable Outcomes

– Strain and sprain only (63%)– Only 1 medical provider (67%)– No emergency room treatment (70%)– PIP claim (72%)– Bill less than $2421 (73%)– Attorney representation (75%)– Accident month November (81%)

Ongoing Research • New dependent variable for expected savings

– Refine model of expected payment – Determine estimated savings from investigations– Generate decision tree based on estimated savings

• Combining variables– Medical provider type– Injury type– Accident quarter (rather than month)

• Examine medical provider/attorney connections• Suggestions?