HABITUAL & NON-HABITUAL RESPONDENTS IN A LONGITUDINAL TELEPHONE STUDY:
DIFFERENCES & IMPLICATIONSP R E S E N TAT I O N F O R A A P O R 2 0 1 6
1
ERAN BEN-PORATH, SSRS
BIANCA DIJULIO, KAISER FAMILY FOUNDATION
MOLLYANN BRODIE, KAISER FAMILY FOUNDATION
JAMIE FIRTH, KAISER FAMILY FOUNDATION
KAISER FAMILY FOUNDATION CALIFORNIA UNINSURED LONGITUDINAL PANEL SURVEY
• Baseline Survey: July-August 2013 (Prior to Open Enrollment)
• 2001 Randomly Selected California Residents:
− Ages 19-64
− Uninsured for at Least 2 Months at the Time of the Baseline Survey
• RDD Landline and Cell Phone, Oversampled Areas with Lower-income Populations
• English and Spanish
• Wave 2: April-June 2014 (After Open Enrollment Period Closed)
• Wave 3: Feb-May 2015 (After 2nd Open Enrollment Closed)
• Wave 4: Feb-June 2016 (After 2nd Open Enrollment Closed)
• Funded in Part by the California Endowment
• All Waves Fielded by SSRS
© SSRS & Kaiser Family Foundation 2016 2
EFFORT TO MAINTAIN COOPERATION RATES
Attempts to Recontact Respondents at Waves 2-4
• Collected Multiple Pieces of Contact Information, Emails, Alternate Phone #S• Incentives to All Respondents, Including Following Up with Those Who Didn’t Cash Their Checks to
Make Sure They Received it• Waves 2-4: Pre-notification Letters with $2 Pre-incentives to Anyone Who Gave Us Their Address
for Incentive• Up to 30 Callback Attempts• Reminders Via Email (if Provided An Email Address), Postcards, Voicemails• Look-up For Alternative Phone Numbers For Numbers That Came Up As Non-working • Continue Re-dialing Non-working• Allowing Respondents to Complete Interviews “On Demand” if They Call in Directly • Web Version of the Survey for People When Several Attempts to Contact Via Phone Were
Unsuccessful
© SSRS & Kaiser Family Foundation 2016 3
FOUR TYPES OF RESPONDENTS (BY COOPERATIVENESS)
I II III IV
Type Dropouts Casual Committed Habitual
Definition Only Wave I Any 2 waves Any 3 waves All 4 waves
N
© SSRS & Kaiser Family Foundation 2016 4
FOUR TYPES OF RESPONDENTS (BY COOPERATIVENESS)
I II III IV
Type Dropouts Casual Committed Habitual
Definition Only Wave I Any 2 waves Any 3 waves All 4 waves
N 560 367 365 709
© SSRS & Kaiser Family Foundation 2016 5
FOUR TYPES OF RESPONDENTS (BY COOPERATIVENESS)
I II III IV
Type Dropouts Casual Committed Habitual
Definition Only Wave I Any 2 waves Any 3 waves All 4 waves
N 560 367 365 709
© SSRS & Kaiser Family Foundation 2016 6
• Assumption: Non-random differences between types.• Systematic non-response
DEMOGRAPHIC DIFFERENCES BETWEEN RESPONDENTS TYPES
55.4%
50.7%
45.5%
42.0%
44.6%
49.3%
54.5%
58.0%
DROPOUTS
CASUAL
COMMITED
HABITUAL
GENDER
m f
© SSRS & Kaiser Family Foundation 2016 7
DEMOGRAPHIC DIFFERENCES BETWEEN RESPONDENTS TYPES
31.4%
23.2%
20.3%
16.4%
23.9%
21.0%
22.2%
18.6%
19.5%
27.2%
22.2%
19.9%
25.2%
28.3%
35.3%
45.1%
DROPOUTS
CASUAL
COMMITED
HABITUAL
AGE
18-29 30-39 40-49 50+
© SSRS & Kaiser Family Foundation 2016 8
DEMOGRAPHIC DIFFERENCES BETWEEN RESPONDENTS TYPES
32.9%
29.7%
32.9%
22.0%
31.8%
32.2%
26.6%
24.5%
21.1%
27.0%
25.2%
34.6%
12.5%
10.6%
14.2%
18.3%
DROPOUTS
CASUAL
COMMITED
HABITUAL
EDUCATION
<HS HS Grad Some Coll Coll+
© SSRS & Kaiser Family Foundation 2016 9
DEMOGRAPHIC DIFFERENCES BETWEEN RESPONDENTS TYPES
OTHER SIGNIFICANT DIFFERENCES
HABITUALS: Significantly More Likely to Have Completed Original
Interview in English than Any Other Type (and Conversely: Least Likely in Spanish)
HABITUALS:Significantly More Likely to Have had a Landline in Their
Homes (At T1)
© SSRS & Kaiser Family Foundation 2016 10
W1 ATTITUDINAL DIFFERENCES BETWEEN RESPONDENTS TYPES
52.0%
50.1%
56.7%
45.8%
21.1%
25.6%
24.9%
33.9%
27.0%
24.3%
18.4%
20.3%
DROPOUTS
CASUAL
COMMITED
HABITUAL
ACA FAVORABILITY
Favorable Unfavorable DK/Ref
© SSRS & Kaiser Family Foundation 2016 11
BEHAVIORAL DIFFERENCES BETWEEN RESPONDENTS TYPES?
47.3%
49.6%
48.2%
48.7%
31.4%
33.0%
31.2%
32.6%
DROPOUTS
CASUAL
COMMITED
HABITUAL
INTENT TO OBTAIN COVERAGE
Will obtain health insurance Will remain uninsured
© SSRS & Kaiser Family Foundation 2016 12
BEHAVIORAL DIFFERENCES BETWEEN RESPONDENTS TYPES?
27.1%
35.1%
35.6%
41.2%
34.8%
29.7%
26.6%
21.2%
DROPOUTS
CASUAL
COMMITED
HABITUAL
PREVIOUS INSURANCE
Employer Never Insured
© SSRS & Kaiser Family Foundation 2016 13
BEHAVIORAL DIFFERENCES BETWEEN RESPONDENTS TYPES?
50.9%
47.9%
53.9%
47.8%
48.9%
41.6%
CASUAL
COMMITED
HABITUAL
W2 COVERAGE
Covered Not Covered
© SSRS & Kaiser Family Foundation 2016 14
BEHAVIORAL DIFFERENCES BETWEEN RESPONDENTS TYPES?
67.3%
65.1%
72.6%
31.7%
34.6%
25.8%
CASUAL
COMMITED
HABITUAL
W3 COVERAGE
Covered Not Covered
© SSRS & Kaiser Family Foundation 2016 15
ASSESSING IMPACT OF HABITUAL RESPONSE/NON-RESPONSE
• The share of 4-wave habituals increases
• Wave 4: prevailing majority of respondents are Habitual
• Assessing impact of nonresponse
• Using W3 data
© SSRS & Kaiser Family Foundation 2016 16
ASSESSING IMPACT OF HABITUAL RESPONSE/NON-RESPONSE
• The share of 4-wave habituals increases
• Wave 4: prevailing majority of respondents are Habitual
• Assessing impact of nonresponse
• Using W3 data
• Propensity Scoring:− Model includes
Demos
Past insurance pattern
− Controlling for demos:
Past insurance is significant
© SSRS & Kaiser Family Foundation 2016 17
INCORPORATING PROPENSITY SCORE INTO WEIGHTING
Mean Propensity Weight
Male 1.137
18 to 29 1.331
50 to 64 0.802
Spanish 1.196
White 0.746
HS or Less 1.138
CPO 1.203
Always Uninsured 1.256
Once Employer 0.872
Inverse of propensity score included in base-weight
© SSRS & Kaiser Family Foundation 2016 18
WEIGHTED OUTCOMES
W3 Habitual Adjustment
Without With
Uninsured 34.5% 35.0%
ESI 14.5% 14.2%
Medi-Cal 33.3% 32.9%
Self Purchased 12.9% 12.9%
With inverse of propensity score included in base-weight
© SSRS & Kaiser Family Foundation 2016 19
CONCLUSION: NEED TO ACCOUNT FOR HABITUAL RESPONDENTS IN PANEL DATA
• Risk posed in these data seems addressed in weighting
• But: Indication of possible association between habitual respondent status and substantive outcome variables
• Weighting to original sample – an option for minimizing this risk
• Nonresponse reduces statistical power in panel research:
• Smaller N
• Greater Design Effect
• Restricted ability to analysis subgroups
• Findings indicate a concurrent need to take action to minimize bias
© SSRS & Kaiser Family Foundation 2016 20
ERAN BEN-PORATH, SSRS@en_bp
BIANCA DIJULIO, KAISER FAMILY FOUNDATION@BeeDiJulio
MOLLYANN BRODIE, KAISER FAMILY FOUNDATION@Mollybrodie
JAMIE FIRTH, KAISER FAMILY FOUNDATION
Top Related