Data-Driven Enrollment: Getting to Know Your Consumers

© 2016 Enroll America | StateOfEnrollment.org

Ed Coleman, National Analytics and Data Director

Molly Warren, Senior Policy Analyst | 05.13.16

Data Driven Enrollment

© 2016 Enroll America | StateOfEnrollment.org 2

1. What we mean by Data Driven Enrollment2. How we Learn What we Learn3. Findings from OE3

1. Consumer Survey Findings2. Increasing Appointment Attendance – Using Text

messages3. Consumer Preferences – Digging into appointment data

from the Connector4. Email Findings5. Fine Message Test Findings

4. Some Planned Future Work5. Discussion + Q&A

Agenda for Today’s Discussion


DEFINING DATA DRIVEN ENROLLMENT


HOW WE LEARN…


Two Ways we Learn

• Start with Data: Take a data set and look to see what kinds of data you have.

• Look for Trends: For example, we have zip code data from the appointment data in the Connector so can look for trends by geography.

Observational learning

• Start with a Question: During OE2, 41% of people failed to attend connector appointment. How can we increase attendance?

• Design a Test: The test should be specifically designed to answer the question

Experimental learning


CONSUMER SURVEYObservational Learning:


• Fielded an analytics survey of EA’s email list• Generated nearly 3000 responses, nearly 4 times as many responses

as past surveys• Allowed us to group consumers into 7 buckets based on

• 2015 insurance status• Whether or not they shopped for insurance in OE3• 2016 insurance status

• Survey asked about • Health care use and health status• Experience shopping for insurance

• Note: The EA email list leans older and poorer is not a representative sample of the general public or all consumers.

Consumer Survey Details


Non Shopping Reasons

Overall Shop Unin Non Shop Unin Shop Exit Non Shop Exit0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Sha

re o

f Res

pond

ents

Who

Fou

nd T

hese

Issu

es to

be

a B

arrie

r to

Get

ting

Insu

ranc

e

“What were the biggest reasons why you chose to not sign up for health insurance in 2016?”


Consumer Confidence

Shop New

Shop Renew

Shop Unin

Shop Exit

Non Shop Unin

Autorenew

Non Shop Exit

Overall

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

15%

19%

38%

36%

17%

8%

23%

20%

52%

50%

24%

33%

23%

43%

26%

43%

33%

32%

37%

31%

61%

49%

52%

37%

Not Confident Somewhat Confident Very Confident

“How confident are you that you made the best decision for you?”


Additional Information and Confidence“What other information or help would have made you feel more confident about your choice?”

Overa

ll

Autoren

ew

Non S

hop E

xit

Non S

hop U

nin

Shop E

xit

Shop N

ew

Shop R

enew

Shop U

nin0%

10%

20%

30%

40%

50%

60%

70%

Yearly Costs Info Coverage InfoInfo on How Insurance Works In-Person Help from an Insurance AgentIn-Person Help from a Non-Profit Communication from an Insurance CompanyOther

Shar

e of

Res

pond

ents

Who

Wou

ld F

ind

Mor

e In

form

atio

n A

bout

the

Follo

win

g H

elpf

ul


Use of In-Person Assistance

Overall Shop Exit Shop New Shop Renew Shop Unin0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Per

cent

of R

espo

nden

ts W

ho U

sed

The

Giv

en

Res

ourc

e

“Did you use any in-person assistance to help choose a health insurance plan?”


Confidence after using Assistance

Not confident Somewhat confident Very confident0%

20%

40%

60%

80%

100%

8%

40%

53%

How confident are you that you chose the best health insurance option for you?

Series1

Response

Per

cent

of R

espo

nses


Appointment Satisfaction

I was confident in my assister’s knowledge about how health insurance works.

After my appointment, I had a clear understanding of the health insurance options available to me.

I was confident in my assister's knowledge about the health insurance plans available to me.

Attending my assister appointment was a good use of my time.

I would recommend in-person assistance to a close friend.

My assister made me feel comfortable during my appointment and understood my personal situation.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

12%

12%

11%

6%

7%

6%

34%

34%

31%

24%

24%

26%

41%

41%

48%

57%

58%

60%

Please rate the following statements about your appointment

Strongly disagree Disagree Neutral Agree Strongly agree


INCREASING ATTENDANCE RATES

Experimental Learning: In-Person Assistance


Increasing Appointment Attendance

• We noticed by looking at the data from OE2 that appointment attendance rates were not great (41% no-show rate)

• Question: How do we lower the no-show rate / increase the appointment attendance rate?

• Answer: Randomly test different types of reminder messages and see if any of them resulted in lower no-show rates


Reminder Messages

1. Standard Message / Control (no additional motivational language): “Remember! Your appointment for free, expert help with your health insurance application is coming up.”

2. Empathy: “Hey there, we know money’s tight. Get all your questions answered at your appointment for free, expert help with your health insurance application.”

3. Bandwagon: “Hey there, lots of people in your neighborhood are getting free, expert help with their health insurance applications. Don’t miss out!”

4. Positive identity: “Thanks for taking charge of your health and your wallet! Remember your appointment for free, expert help with your health insurance application.”


Attendance Rate by Message

Control Empathy Bandwagon Positive identity0%

5%

10%

15%

20%

25%

30%

35%

29.4%

25.5% 26.2%

30.3%

Connector appointment flake rate by text message treatment group (lower bars are better)

Treatment group

13.4%


No Show Rate with Personalization

Control Bandwagon Empathy0%

5%

10%

15%

20%

25%

30%

35%33%

27%25%25%

24%

26%

Connector appointment no show rates with and without first name personalization (lower bars are better)

Without first nameWith first name

24%

10.5%


CONSUMER PREFERENCES – THE CONNECTOR

Observational Learning:


A Scheduling Tool for Consumers


Growth in Connector from OE2 to OE3


Common & Popular Appointment Times in OE3

Percent of appointments filled by day of week and time of day

Number of attended appointments by day of week and time of day

CO

MM

ON

POPU

LAR


Appointment Uptake by Day in OE2 & OE3

sun mon tue wed thu fri sat0%

5%

10%

15%

20%

17%

12% 12%11% 11%

10%

20%

Day of the Week

% of Appts Filled

sun mon tue wed thu fri sat0%

5%

10%

15%

20%

9%10% 11%

10% 9%9%

14%

Day of the Week

Saturday appointments were the most popular in both OE2 and OE3

Percent of Appointments Filled – OE2 Percent of Appointments Filled – OE3


Attendance Rates by Appointment Type in OE3


Distance Traveled to Appointments in OE2 & OE3

% of Appointments0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

37%

4%7%

9% 8%6%

10%

7%

12%

<1 1-2 2-3 3-4 4-5 5-6 6-8 8-10 >10

Miles Traveled to Appointment - OE2

% of Appointments0%

5%

10%

15%

20%

25%

30%

35%

40%

45%41%

3%

6% 7% 6% 6%

10%

6%

15%

<1 1-2 2-3 3-4 4-5 5-6 6-8 8-10 > 10

Miles Traveled to Appointment- OE3

On average, consumers traveled 4.6 miles to their appointment in OE3


No Show Rate by Appointment Distance in OE3

<1 1-2 2-3 3-4 4-5 5-6 6-8 8-10 >100%

5%

10%

15%

20%

25%

30%

35%

40%

32%

36%33% 34%

31% 32%29%

27%

22%

Miles Traveled to Appointment

No S

how

rate


More in Recently Released Issue Brief!


EMAIL FINDINGSExperimental Learning:


Email Engagement Rate by Source

0%

4%

8%

12%

16%

20%

4.6%

18%

Email open rate

0.0%

1.0%

2.0%

3.0%

0.27%

3.3%

Email click rate

All emails

Field-collected emails


Email A/B Testing - Timing

Adding “monthly” in the subject line resulted in 23% fewer unique opens.

(statistically significant).

Adding “monthly” in the body of the email did not affect the number of

clicks among those who opened the email.

Sent November 15, 2015

Unique opens

Unique clicks

Clicks among opens

Unsubscribes

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Lower monthly Obamacare costs Lower Obamacare costs

****

****

***

Significance levels: * p < .05, ** p < .005, *** p < .0005, **** p<.00005

A B


FINE MESSAGE TESTINGExperimental Learning:


Fine Message Testing on Calculator

• Test was designed and embedded on the Get Covered Calculator

• Users receive estimated eligibility and monthly insurance cost

• During the test, users randomly assigned to receive one of four messages on the fine.


Enrollment by Cost Estimate & Test Group in OE3

Total $99 or less $100-199 $200-299 $300-399 More than $400

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%ControlSimpleExplainDetailed

Calculator cost estimate

"Enroll online" click rate


Enrollment by Fine Estimate & Test Group in OE3

$695 $700-1000 $1000-1500 More than $15000%

5%

10%

15%

20%

25%

30%

35%N = 8755

N = 1716

N = 4579

N = 3534

No detailed fine amount

Detailed fine amount

Estimated Fine Amount

"Enroll online" click rate


• Consumers in the control group (which received no message about the fine) were slightly more likely to click “Enroll Online” than those who received a message on the fine with their calculator results.

• The finding does not mean we should stop talking about the fine.

• Combined with previous fine test results and other research, such as the Perry Undem surveys, this finding suggests that language on the fine is important to motivate certain consumers at certain times, but not universally.

• Enroll America recommends messaging the benefits of enrollment to motivate consumers, such as drawing attention to low cost premiums and availability of financial help, rather than focusing only on the penalty of not enrolling.

Main Takeaways


FUTURE WORK


• Looking for connections between earned media and in-person assistance interest

• Looking at consumer behavior though our new (in OE3) Plan Explorer tool

Future work


Q + A + DISCUSSION

Data-Driven Enrollment: Getting to Know Your Consumers

Healthcare

Transcript of Data-Driven Enrollment: Getting to Know Your Consumers