Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results
description
Transcript of Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results
Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results
Mario Callegaro, Tim Macer
Mobile Phone Penetration Up
Rules of Thumb
No horizontal scrollingVertical scrolling OK
Avoid long listsEspecially in check all that apply
Situation FluidAs Tablet become Popular
Depends on the Platform
Platform Considerations
Need to Test on Multiple Platforms
Apple (iPhone/iPad) can’t implement Flash
Default on all phones is to not enable Java
Useful Paradata
UserAgentStringDevice\Model
Operating System
Screen Resolution
Fonts
“Can You See It Now? Good”Usability Testing of a Mobile Health Application
Sarah Cook, Rita Sembajwe, Emily Geisen, Barbara Massoudi
New Way to Do a Diary
Benefits
Immediate Results
Cost Effective
Create Easy to Use Dashboard
Usability Suggestions
Don’t scroll vertically on Select All
Make it easy to trace any sliding
Hard to video what they do
Mobile Phone Effects at Event-Based Sampling
Dan Williams
Case Study
Three Modes of Collection
WebMost Popular
Not all on Mobile Device
IVRCapture Older Population
SMSImmediate Response
Younger Respondents
Are you who you say you are? Using a Multisource Cross-validation Methodology for
Panel Membership Information.
Kumar Rao
Real, Unique, and Engaged
3rd Party Database ValidationInclude Demographics
Use Multiple Databases
Results
Cost could be worth the extra
All more likely to be established households
False Positives Too High
Still Important Part of Process
Differential Sampling Based on Historical Individual-Level Data in Online Panels
Richard Kelly
Quota Sampling
Way to Deal with Non-ResponseDidn’t Know Demographics
More Efficient to Screen Out
Just Transferred Over
to Online
Differential Sampling
Know the Demographics
Know the Response Rates
Oversample those Hard to Reach
More Efficient and Cost Effective
Designing Questions for Web Surveys: Effects of Check-List, Check-All, and Stand-Alone Response
Formats on Survey Reports and Data Quality
Jennifer Dykema, Nora Cate Schaeffer, Jeremy Beach, Vicki Lein, and Brendan Day
Three Types Web Designs
Check-ListMore Items Selected
Check-AllLower Break-offs
Stand AloneLess Primacy Effect
Category Selection Probing in Online Access Panels
Dorothée Behr, Lars Kaczmirek,
Michael Braun, Wolfgang Bandilla
Cognitive Testing OE
Face-to-Face too Expensive
Online TestingProbing Open Ends
Community vs PanelMore chatty?
Results
Topic Trumps Source
Use Communities Built Around the Topic
Face-to-Face More Involved
Response Quantity, Response Quality, and Costs of Building an Online Panel via Social Contacts
Vera Toepoel
Snowball Recruiting
No Online Panel in NL RepresentativeRequires More Commitment
Try Refer a Friend Program
Use Network Theory
Results
Snow Never RolledOnly got 120 recruits
Don’t Use Students
Incentives not Worth the Cost
Representativeness
The Use of Web Panels to Characterize Rare Conditions
John Boyle
Hard to Reach Population
Only 23 in a sample of 10,000 HH
Costs High
Variance Too High
Important Diseases
Clean the Online Data
Certain Improbable Conditions
Speeders
Straightliners
Results In Line
Prevalence In Line
Treatments Numbers Good
Got Much More Sample Size
Cost Less
Measuring Intent to Participate and Participation in the 2010 Census and Their Correlates and Trends:
Comparisons of RDD Telephone and Non-probability Sample Internet Survey Data
Josh Pasek and Jon Krosnick
Intent to Complete Census
Better Demographics Compositions
Intent Numbers Varied
Predictors for Intent to Complete Different
Trends Also Different
Can a Non-Probability Sample Ever be Useful for Representing a Population?: Comparing Probability and Non-Probability Samples of Recent College Graduates
Cliff Zukin, Jessica Godofsky, Carl Van Horn, Wendy Mansfield, and J. Micheal Dennis
Comparing Sampling
Probability Samples have a Prob Theory
Can’t Intelligently Trade Off Error
Compare KN Panel to volunteer Panel
Recent Graduates
Results
Differences between probability and non-probability panel
No mode effects or questionnaire effects
Differences mitigated a lot when weighting for other non-quota variables