Surveys with Mixed Samples and Modes - David Dutwin, Ph.D.
40
-
Upload
ssrs-market-research -
Category
Data & Analytics
-
view
41 -
download
0
Transcript of Surveys with Mixed Samples and Modes - David Dutwin, Ph.D.
research. refined.Sometimes Hybrids Work…
research. refined.…and Sometimes They Don’t
research. refined.
• Telephone surveys appear remarkably resistant to systematic bias, despite low response rates.
• However, costs continue to rise and response rates continue to drop.
Why Talk About Mixed Samples & Modes?
research. refined.Typical Drivers of Mixed Sample/Mode Designs
research. refined.
• CDSF file• Enumerated/supplemented• Empaneled
• Single/Dual Frame• Telephone Recontacts• Empaneled
• Full Universe Samples• Internet Empaneled
• In-Person w/ w/o Interviewer
• Telephone w/ w/o Interviewer
• Internet
Samples & Modes
research. refined.Options Within Samples
• Appended Sample from available sources
• Listed sample of any kind with incomplete coverage
• High incidence areas or other geographically selected subsamples
• Internet could/should be considered incomplete at least due to coverage area
• Etc.
research. refined.
• Can have any combination of inbound phones, outbound phones, mail, and internet
• A single sample frame
Popular Mixed Samples/Modes Designs
• 2 samples• 2 modes
• 2 samples• 1 mode • 2 samples
• 1 mode
• multiple samples
• 1 mode• multiple
samples• 2 modes
research. refined.Problems of Combining Multiple Samples
The weight of academic research continues to find in nonprobability samples 1) greatersystematic non-response (bias) and 2) greater variability in the size of bias than in probabilitysamples.
One mode has more measurement error than another on variables of interest. However, in asingle sample study, mode becomes a sample type, potentially confounding with measurementerror.
research. refined.Problem 1: Probability & Nonprobability
research. refined.Problem 1: Probability & Nonprobability
0 5 10 15 20
Luth Web Unweighted
Research Now Web Unweighted
SSRS Omnibus Cell Phones…
SSRS Telephone Omnibus…
SportsPoll Telephone Unweighted
NHIS Unweighted
Luth Web Raked
Research Now Web Raked
Research Now Propensity Weighted
Research Now Propensity Weighted…
Research Now Matched
Research Now Matched and Raked
Luth Web Matched
Luth Web Matched and Raked
Cell Phones Raked
SSRS Omnibus Telephone Raked
SportsPoll Raked
NHIS Weighted
Race within Education
Age within Education
Region within Education
Race within Age
Education within Age
Region within Age
Age within Race
Region within Race
Education within Race
Race within Region
Age within Region
Education within Region
research. refined.Problem 2: Multiple Modes
• Two samples, different modes in each• One sample, sequential or concurrent mixed-mode offering• Multiple time points via panel or cross-sectional
1) Media-related factors: who has control over the Q and A2) Factors influencing the information transmission: visual/auditory, verbal, nonverbal, paralinguistic
differences and visual/graphical/layout 3) Interviewer effects: Privacy, locus of control, social desirability, etc.
1) Less social desirability2) More missing data3) Less extremeness (evidence weak)4) Inconsistent effects with regard to recency/primacy
research. refined.
•
•
•
•
•
•
•
Problem 2 & 3
research. refined.Correction of Sample & Nonresponse Error
research. refined.Typical Problems to Solve
•
•
•
•
•
•
•
•
research. refined.
Percent of Jews Percent of Sample Weight
Client List 40% 65% 0.62
Name Targeted 10% 15% 0.67
Residual 50% 20% 2.50
•
•
•
Standard Base Weighting for Selected & Residual Designs
research. refined.Traditional Weighting•
•
•
•
research. refined.Propensity Modeling & Weighting
research. refined.Propensity Modeling & Weighting
research. refined.Propensity Modeling & Weighting
research. refined.Variant Models
research. refined.Propensity Stratification Weighting
research. refined.Early Webographics
research. refined.
research. refined.
•
•
•
•
•
research. refined.The Alternative: Calibration
•
•
research. refined.Calibration Applications - DiSogra, 2013
research. refined.Calibration Applications - DiSogra, 2013
research. refined.Case Study 1: Centris Survey
research. refined.Case Study 1: Centris Survey
research. refined.Case Study 1: Centris Survey
research. refined.Case Study 1: Centris Survey
research. refined.Case Study 1: Centris Survey
research. refined.Case Study 1: Centris Survey
research. refined.Case Study 2: Adjusting for Multiple
Waves/Recontact Samples
research. refined.Case Study 2: Adjusting for Multiple Waves or
Recontact Sample
research. refined.Problem 3: Mode Effects
research. refined.Problem 3: Mode Effects
•
•
•
research. refined.References
thank you for your time
