Post on 16-Feb-2016
description
Timing Isn’t Everything, But Money TalksHow to encourage a face-to-face household panel to go online?
University of Essex, July 2013
Gerry NicolaasCarl Cullinane
2
ContentsBackgroundDesign of ExperimentResults Summary of Results and Conclusions
Background
1.
4
BackgroundSwitch to mixed mode data collection at wave 7 of Understanding SocietyPotential for cost savingsPotential for reducing attrition
5
ContextLongitudinal household panelFace-to-face interviewing of all household members at waves 1 to 6Greatest potential for reducing data collection costs when an interviewer does not have to visit the householdPrevious experiment mixing telephone & face-to-face:
Costs can be reduced BUT response rates suffer
6
Innovation Panel – wave 5Vehicle for methodological development & testing About 2,500 individuals in 1,500 householdsMain objective of IP5 =
Determine whether it is possible to reduce costs by mixing web questionnaires and face-to-face interviews without sacrificing data quality
Sequential mixed mode design starting with web
77
IP5 Design
Experimental group
F2F phase, web open
Web only phase
Control group F2F only phaseNo web
8
Research question Is it possible to boost the proportion of whole households completing web questionnaires by
Timing the arrival of the invitation to go online?
Offering a web bonus?
Design of experiment within mixed-mode sample
2.
10
Design of experiment within mixed-mode sample(1) Timing of invitation to go online Random allocation of households to:
Letter (+ email) to arrive on Friday Letter (+ email) to arrive on Monday
Reminder letters (+ emails) sent 2 and 4 days later
(2) Web bonus Random allocation of households to:
No web bonus £5 per household member conditional on all completing online questionnaire
11
Design of experiment within mixed-mode sample
270 household
s
265 household
s266
households
276 household
s
No web bonus
Cond. £5 per hhld
member
Invite arrival on
Friday
Invite arrival on Monday
Results
3.
13
Experimental effects on Web Response
Base: IP5 WEB Sample (n=1077)
21.4 23.7
0
20
40
60
80
100
Monday Friday
Full Household Web Response (%)
14
Experimental effects on Web Response
Base: IP5 WEB Sample (n=1077)
21.4 23.7 20.4 24.7
0
20
40
60
80
100
Monday Friday No Bonus Bonus
Full Household Web Response (%)
15
Effect of Bonus by Sample Type
Base: IP5 WEB Sample (n=1077)
188.6
3430.5
9.4
33
0
20
40
60
80
100
W4 Responding W4 Non responding Refreshment
No bonus Bonus
16
Effect of Bonus by Upfront Incentives
Base: IP5 WEB Sample (n=1077)
13.5 17.9
37.4
18.8 22.6
40.2
0
20
40
60
80
100
£5 £10 £20+
No bonus Bonus
1717
Unconditional Incentives
Original Sample
Refreshment Sample
£5
£10
£10
£30£20
18
Effect of Bonus by Children in Household
Base: IP5 WEB Sample (n=1077)
22.8 20.6 16.43.7
22.833.3 35.4
18.2
0
20
40
60
80
100
None One Two Three +
No bonus Bonus
19
Effect of Bonus by Advance Mailing
Base: IP5 WEB Sample (n=1077)
5.1
36.3
6.5
43.8
0
20
40
60
80
100
Postal Valid email
No bonus Bonus
20
Effect of timing by Advance Mailing
Base: IP5 WEB Sample (n=1077)
4.8
38.3
6.8
41.9
0
20
40
60
80
100
Postal Valid email
Monday Friday
21
Experimental effects on Final Response
Base: IP5 WEB Sample (n=1077)
51.9 54.7
23.7 18.1
0
20
40
60
80
100
Monday Friday
Full household Partial household
75.6 72.8
22
Experimental effects on Final Response
Base: IP5 WEB Sample (n=1077)
51.9 54.7 53.6 53
23.7 18.1 20.4 21.4
0
20
40
60
80
100
Monday Friday No Bonus Bonus
Full household Partial household
75.6 72.8 74 74.4
23
Estimation of cost savingsWeb bonus of £5 compared to no web bonus:Small reduction in travel and mileage costs15% reduction in interviewer fees Offsetting the cost of web bonuses reduces the saving in interviewer fees to less than 5%
24
Summary of results and conclusions 4
.
25
Summary of resultsFriday mailing had a small but diluted effectWeb bonus increased full household web responseEffect of bonus varied by a number of factors, e.g.
Diminishing returns with larger upfront incentives Greater effect for households with children Greater effect when household contacted by email
Web bonus did not increase final response rateBut potential for cost savings
26
LimitationsSmall sample sizesConfounding of sample type and value of unconditional incentives sent with advance letterEstimation of costs
27
Points for discussionTiming to be looked at further?Potential for greater cost savings through targetingLarge upfront incentives vs conditional web bonus- costs
If you want further information or would like to contact the author,Carl CullinaneT. 020 7549 7158E. carl.cullinane@natcen.ac.ukVisit us online, natcen.ac.uk
Thank you