Practical Predictive Modeling - ACHS - Home · Wait a second, haven’t actuaries always done...

Copyright © 2014 by The Hartford . All rights reserved. No part of this document may be reproduced, published or posted without the permission of The Hartford.

Practical Predictive Modeling

Shane Barnes, FCAS

Michael Ewald, FSA, CFA, CERA

May 21st, 2014

Copyright © 2014 by The Hartford . All rights reserved. No part of this document may be reproduced, published or posted without the permission of The Hartford. .

Agenda

1. What is predictive modeling and how should actuaries think about it?

2. Current state in P&C market

3. Current state in Life market

4. Predictive Modeling Case studies & Building a Model

5. Challenges

6. Business Interface


How do you think about predictive modeling? Is it actuarial voodoo?

Model

Is this how you think about modeling?

What about our business partners?

When the model is finished we just

expect it to be right and everyone is

happy. Some people feel it is not that

necessary to understand the black box.


Wait a second, haven’t actuaries always done predictive modeling? Couldn’t

mortality, policyholder behavior, or claim incidence be a predictive model?

What was on the previous slide is not predictive modeling. What is the definition of predictive modeling?

Predictive modeling is the ability to predict future outcomes using various

statistical techniques.


Predictive modeling is not a black box, but rather a more sophisticated way to do our analytics. This is how we should look at models.

The only tricky part of modeling is

learning how to use the tools

effectively.

Data

Assumptions

Modeling

Procedure:

SAS, R,

Excel

Model Output:

Actual = Expected

Data is vital when constructing a

model, but this isn’t different than a

typical actuarial analysis. Being explicit

with your assumptions is a vital piece

in building out the analysis

Being able to explain a model

effectively is an important step in the

modeling process. One reason why

people think models are a black box is

because we don’t explain the

preceding steps to them.


Let’s break this down into a model we all know: Mortality Rate Study

A mortality analysis looks at historical

data to determine future trends

2007-2013

Data

Assumptions

Mortality Process:

Use one-way and

two-way analyses.

Final mortality

tables by various

factors.

The data can be organized in various

fashions such as product type.

The final product is the mortality that

is presented to the business. Do our

business partners feel this type of an

analysis is a black box? Why not?


Current State in P&C Market

• Min Bias Procedure was used for insurance pricing

• GLMs became increasingly popular in the 1990s with computing power

• GLMs became the norm in the 2000s for most of the insurance products with personal lines being the leader

Prelude

• Companies are developing predictive modeling capabilities for commercial lines

• Price optimization algorithms being developed for personal lines

• Models being built to improve claim processes

Current State

• Predictive modeling branching outside of core product base

• Price Optimization being developed for commercial lines

• Incorporate modeling to better understand buying behavior and lifetime value of a customer

• Models being built outside of traditional pricing and claims

Future State


CURRENT STATE - LIFE


2012 SOA SURVEY OF LIFE COMPANIES

9

Currently Using or

Considering Not Considering

Term/Whole/Universal 45%-50% 50%-55%

Variable Life 22% 78%

Other 17% 83%

SOA Survey on Fully Underwritten Life Insurance:

Currently

Use Plan to Use

Don’t Plan to

Use

Personal Auto 80% 15% 5%

Homeowners 62% 33% 5%

Source: Stoll, Brian, & Southwood, Klayton. March 2013. 2013 Predictive Modeling Benchmarking Survey. Retrieved fromhttp://www.towerswatson.com/en-US/Insights/Newsletters/Americas/americas-insights/2014/predictive-modeling-usage-for-property-casualty-insurers-grows

Source: Society of Actuaries. January 2012. Report of the Society of Actuaries Predictive Modeling Survey Subcommittee. Retrieved fromhttp://www.soa.org/research/experience-study/bus-practice-surveys/research-2012-02-predictive.aspx

Towers Watson Survey on P&C Usage:


2012 SOA SURVEY OF LIFE COMPANIES

10

Yes Maybe No

Cross-selling to Current Customers 18% 25% 57%

Up-selling to Current Customers 18% 24% 59%

Lead Generation 14% 26% 60%

Target Marketing 20% 34% 46%

Level of Future Sales 8% 31% 61%

Percentage of Companies Using PM for Marketing:

Source: Society of Actuaries. January 2012. Report of the Society of Actuaries Predictive Modeling Survey Subcommittee. Retrieved fromhttp://www.soa.org/research/experience-study/bus-practice-surveys/research-2012-02-predictive.aspx


PREDICTIVE ANALYTICS IN THE HEADLINES

• Lincoln Financial Press Release:

– “… Lincoln has implemented Predictive Modeling in its analysis of linkage between lapse behavior and variables such as age, gender, and

policy size and duration.”

• CIGNA Press Release:

– “The Cigna study found that a combination of predictive analytics and a nurse health advocate-led intervention can produce a measurable

reduction in future disabling illness or injury incidents…”

• MassMutual Job Posting 50354868:

– “The … (FPD) Consultant … will be responsible for developing a Global

Fraud Prevention and Detection strategy … utilizing fraud surveillance technology, predictive analytical models and relational databases.”

11


BUILDING A MODEL & CASE STUDY


MODELING REQUIREMENTS

13

Modeling Expertise

Business Expertise

Data Expertise

Modeling Team

• Data, data, data

• The more volatile the response, the more data you need

• Data availability can be the biggest hurdle

• Time horizon – not too long, not too short

• Awareness of system limitations

• Ability to translate into business action

• Modeling software


THE MODELING PROCESS

14

Data Preparation

•Check validity of data – look for data errors

•Transform variables

•Review one-way cuts, check correlation matrix.

•Separate data into model dataset (Train) and test dataset (Test) – ENSURE INDEPENDENCE

Model Building

•Add main effects (individual variables)

•Evaluate interactions (how two variables impact one another)

•Test control variables

Model Validation

•Check model’s fit on test dataset

•Out of time sampling

•Backwards regression

Selections & Dislocation

•Use business intuition to adjust factors

•For variables with too little data to model, use business intuition to develop factors

•Assess the change from old methodology to new methodology (especially in pricing)

Implement & Monitor

•Roll out model for business use

• It is critical to continually assess the models performance and determine when refresh is necessaryBU

SIN

ES

S I

NP

UT

AT

AL

L S

TA

GE

S


DATA PREPARATION – THE 80%

15

It is critical to understand the data. Source systems may be fed manually. Data is probably not entered with you in mind.


FINDING THE TRUE IMPACT OF VARIABLES

16

3 Months 6 Months

← L

ow

F

req

ue

nc

y

H

igh

→

Observed vs. Modeled- Finding the True Slope

Observed Slope

Modeled Slope

16% Difference

One-way analysis would yield a much flatter incidence slope. The true effect

of EP on incidence is much steeper!


INTERACTIONS – ASSESSING POTENTIALS

17

Actual incidence for younger females is much higher than predicted.

Without an interaction in the model, we do not appropriately differentiate younger males and females.

←L

ow

F

req

uen

cy

H

igh→

← Young Age Old →

A to E - Age x Gender (Before Interaction)

Male - Actual

Male - Fitted

Female - Actual

Female - Fitted


INTERACTIONS – FITTING THE INTERACTION

18

After adding the age & gender interaction we are able to more accurately predict the incidence for younger individuals.

←L

ow

F

req

uen

cy

H

igh→

← Young Age Old →

A to E - Age x Gender (After Interaction)

Male - Actual

Male - Fitted

Female - Actual

Female - Fitted


←L

ow

F

req

uen

cy

H

igh→

← More Rural More Urban →

Urbanicity - The Impact of Other Variables

Actual

Predicted

WHAT IS A VARIBLES IMPACT, IF ANY?

19

The other variables in the model do a very good job of predicting rural vs. urban areas. Most rural incidence is double that of most urban incidence.


ASSESSING THE POWER OF EACH VARIABLE

20

Variable # % Deviance Cumulative %

Variable 1 49.90% 49.90%

Variable 2 14.62% 64.53%

Variable 3 7.48% 72.00%

Variable 4 7.46% 79.46%

Variable 5 4.87% 84.33%

Variable 6 3.23% 87.57%

Variable 7 2.37% 89.93%

Variable 8 2.20% 92.14%

Variable 9 1.91% 94.04%

Variable 10 1.71% 95.75%

… … …

LTD Incidence Rate Model – Power of Each Variable

One variable provides 50% of the predictive power of the model. Later variables only add slight predictive power.


Decile 1 Decile 2 Decile 3 Decile 4 Decile 5 Decile 6 Decile 7 Decile 8 Decile 9 Decile 10

←U

nd

erc

harg

e

O

verc

harg

e

→

← LOW RISK HIGH RISK →

Actual vs. Expected LTD Claim Cost by Risk Decile

Old Manual / Actual New Manual / Actual

ACTUAL TO EXPECTED – OLD VERSUS NEW METHOD

21

A/E = 1.0

Flatter line indicates more appropriate pricing of risk. The old manual was overcharging for lowest risk cases and undercharging for highest risk cases.


BUSINESS INTERFACE


There is a balance between advancing the science the adoption of the new techniques.

Actuarial Advancement

Business Readiness

Is the data set up the

right way to be able to

build a good model?

How do you develop

the expertise to be

able to build a model?

Does the business have

the appetite to explore

the unexplored?

Do the benefits

outweigh the cost?

Is the investment in

people and money

worth sacrificing

business deliverables?


Business Interface: Know your customer

Understand the Business

Understand how business is conducted

How/what data is captured?

What variables can we use?

Understand the business

issues

Ask questions, lots of questions

Even though you an do cool stuff

doesn’t mean you should

Don’t waste valuable time

Talk the language

Allows you to become a trusted

advisor

People have more confidence in your

work


Business Interface: Clear and frequent communication!

Communication

Model Champion

Transparency

Regular Meetings

Over-

Communicate

Have one person

who you trust and

who can be your “go-

to” person and

support your cause

This is vital, the

more you give

them, the better

they feel

Err on the side of too

much information,

they will let you know

when they are good.

Having regular

meetings with your

key constituent

will present clarity

in your work.

Practical Predictive Modeling - ACHS - Home · Wait a second, haven’t actuaries always done...

Documents

Transcript of Practical Predictive Modeling - ACHS - Home · Wait a second, haven’t actuaries always done...