Successful Online Surveys

PresentersKim Perkins, Timothy C. Lisk, & Justin C. Mary

DiscussantDale E. Berger

Claremont Graduate University

WPA 2010, Cancun, Mexico

Survey Survivor:How NOT to get Voted Off the Island

Kim Perkins & Timothy C. LiskClaremont Graduate University

WPA 2010, Cancun

OVERVIEW Intro Comparison of Services

FreePaidCase-by-case

Tribal Council Tips & Resources

INTRO TO ONLINE SURVEYS Strengths

Easy to administerNo data entry!Bias-free random assignment

WeaknessesTake longer to createSample issuesWho owns the data? Inconsistent formattingFeatures change all the time

JARGON Skip logic & branching

Earlier responses determine later items IF yes THAN item #4

RandomizationQuestion, answer, pageReduces order effectsNot random assignment

IP addressSimilar to a cell phone number Identifies one computer at one point in time

FREEBIE MATRIX

Questions

Responses

Skip Logic

Export to…

Randomization

T&K Rating

Survey Monkey

10 100/survey

-- -- Yes

Zoomerang 12 100/survey

-- -- --

Question Pro

10 Unlimited -- Excel --

Lime Service Unlimited 25/mo Yes Excel / SPSS

Yes

Survey Gizmo

Unlimited 250/mo -- Excel Yes

FREE SERVICES Best for Newbies: SurveyGizmo

Unlimited surveys / questions250 responses per month

Best for Experts: LimeSurveyLocal or remote hostingAll features available

$$$ MATRIXCost Questio

nsRespons

esSkip Logi

cExport

to…Randomizati

onT&K

RatingSurvey Monkey

$19.95/mo Unlimited 1,000/mo Yes Excel Yes

Zoomerang

$199/year Unlimited Unlimited Yes Excel Yes

Question Pro

$15/mo Unlimited Unlimited -- Excel --

Lime Service

Per response: 100=$10

1000=$60

Unlimited Unlimited ($)

Yes Excel / SPSS

Yes

SurveyGizmo

$19/mo Unlimited 1,000/mo Basic Excel Yes

“Surveys for Dummies”Or at least people who would rather not

think about this too much Free is useless, but $200/yr:

“One-click” to import items straight from Word

Automatic crosstabs and chartsReal-time widgets for web page placement

$899/yr – “Expert team” will write your survey

Very common Good real-time data display

Useful to display data to stakeholders $200/year package is cheaper than

$19.95 monthly, with no limits on # of responses

SURVEY MONKEY

SURVEY MONKEY

SURVEY MONKEY - CHART

Easy and powerful Skip logic & piping ($$$) “Respondent Anonymity Assurance” ($$

$) IRB friendly

No longer offer academic institution licensing

Open source with high-end features Can download entire package to your own site & use it

free Default: Does not pick up IP addresses Assessments Tokens for matching participants in pre-post design

Pay by the response as you go Packages of $.04 to $.10 per response Even free accounts can use advanced features

Clunky design interface Perplexing web-developer logic & vocabulary

Response set = Label set Skip logic = Conditions Matrix-style questions = Answers

LIME SURVEY

LIME SURVEY

LIME – LABEL SETS

LIME – DATA VIEW

LIME - CHART

Anonymous IP / SSL encryption ($$$) Solution? Free student account!

All features unlockedNo formal tech support .edu addresses only

GIS summary Random assignment

A/B branching Ability to combine surveys

WILD CARD: $25/mo IRB-friendly SSL encryption Account purged 14 days after

deactivation Super Uptight Critique

Company still collects IP addressesUltimately, data is password secured

“However, should interception of encrypted data occur, that data could not be decoded without the

unique encryption key that is held only by PsychData.”

WPA COUNCIL VOTE Defend the providers! Raise your hand when your favorite

provider is called.

AND THE WINNER IS…SurveyGizmo! Strengths

Easy to useMost key features

available Weaknesses

Basic logic & branching IP address tracking

Runner-up: LimeSurvey

TIPS & RESOURCES URL shorteners Random assignment Branching / logic maps “Hidden” help files IRB application help

RECAP Comparison of Services

FreePaidCase-by-case

SurveyGizmo Tips & Resources