Introduction to UX Research: Designing Surveys That Don't Suck!
-
date post
17-Oct-2014 -
Category
Design
-
view
4.638 -
download
1
description
Transcript of Introduction to UX Research: Designing Surveys That Don't Suck!
Will Evans
Yana Kuchirko
"They've done studies, you know. 60% of the time, it works every time. ”
- Brian Fantana
04/07/23 2
04/07/23 3
Surveys Aren’t Objective
They are created by people who are biased and analyze their results based on their own (mis)perceptions.
Epistemological Dead End?
No. Awareness that the researcher plays an integral role in the process of measuring any given phenomena by deciding how to measure is key.
There are ways to minimize researcher bias by creating better questions.04/07/23 4
Methods are not “just methods”
04/07/23 5
How you measure what you are studying shapes what you find.
Let’s say you are studying…
04/07/23 6
How often teenagers use your website
You might ask teenagers…
“How often do you my website?”
a.Very rarelyb.Rarelyc.Occasionallyd.Frequentlye.Very frequently
04/07/23 7
Response options source: http://www.dataguru.org
You might ask teenagers…
“How often do you my website?”
a.Very rarelyb.Rarelyc.Occasionallyd.Frequentlye.Very frequently
04/07/23 8
Response options source: http://www.dataguru.org
And what’s wrong with these????
You might ask teenagers…
“How often do you my website?”
a.Very rarelyb.Rarelyc.Occasionallyd.Frequentlye.Very frequently
04/07/23 9
Response options source: http://www.dataguru.org
And what’s wrong with these????
SUBJECTIVITY!!!
“Hmmm, for me “frequently” really means….”
A few times a week A few times a day
So what are we really measuring? NOISE.
04/07/23 10
What is “noise”?
What we observe…
04/07/23 11
We WANT this!We WANT this!
We DON’T WANT this!We DON’T WANT this!
04/07/23 12
What is “noise”?
Let’s really stretch our thinking a bit here to provide a more concrete understanding of “noise” in your data.
Imagine you have AT&T phone service and you’re trying to make a call. The signal isn’t clearly going through and you end up hearing everything but the other person’s voice. Frustrating, right? Thank God this is only a pretend scenario.
04/07/23 13
An imaginary scenario
How is AT&T related to “Noise”?
When you develop bad questions, you don’t “hear” the message of your data clearly.
04/07/23 14
So…why is “noise” bad???
• Badly structured and poorly worded questions that obfuscate meaning for participants provide bad data.
• Bad data– Doesn’t answer your research question–Makes it difficult to interpret results– Is pretty much useless.
04/07/23 15
More bad examples of frequently used response
optionsa. Completely satisfiedb. Very satisfiedc. Fairly well satisfiedd. Somewhat dissatisfiede. Very dissatisfied
a. Totally likeb. Very much likec. Moderately liked. Somewhat likee. Not like
What’s the difference between “fairly well” and “somewhat”?
What’s the difference between “moderately like” and “somewhat like”?
04/07/23 16
a. Completely satisfiedb. Very satisfiedc. Fairly well satisfiedd. Somewhat dissatisfiede. Very dissatisfied
a. Totally likeb. Very much likec. Moderately liked. Somewhat likee. Not like
What’s the difference between “fairly well” and “somewhat”?
What’s the difference between “moderately like” and “somewhat like”?
Semantics04/07/23 17
More bad examples of frequently used response
options
Words like….
• Sometimes• Often• Moderately• Very• Not very much
Are subjective = they mean different things for different people
04/07/23 18
A better alternative?
Actually asking people how often they do certain activities.
04/07/23 19
In order to gather objective data…
…your questions must mean the same thing for everyone.
How often do you use my website?a.Neverb.A few times a yearc.Once a monthd.2-3 times a monthe.Once a weekf.A few times a weekg.Every day
“Never” means never for everyone!04/07/23 20
Importance of “Anchors”
Responses options serve as “anchors” for each question, determining the “location” of each responses as qualitatively distinct from the other.
04/07/23 21
Responses options serve as “anchors” for each question, determining the “location” of each responses as qualitatively distinct from the other.
04/07/23 22
Importance of “Anchors”
Strongly disagree Disagree Neutral Agree
StronglyAgree
(Ideal) Response Symmetry
• Good questions aim to have symmetrical quantitative/qualitative distance between anchors a. Neverb. A few times a yearc. Once a monthd. 2-3 times a monthe. Once a weekf. A few times a weekg. Every day
The distance between “never” and “a few times a year” is proportionate to “every day” and “a few times a week”.
04/07/23 23
Adherence to Logic & Linearity
People are accustomed to paradigms that are intuitive, and often “linear”.
Anchors should not be the exception.04/07/23 24
Intuitive Anchor Directions
Never Always
Disagree Agree
Very poor Very good
Not very important
Very Important
04/07/23 25
Value of Clarity
If questions are clear and concise, participants would spend less time analyzing the questions themselves and more time on answering them. Mystic Arts, LLC
04/07/23 26
So do good questions guarantee valid results?
• No. But good questions offer more assurance that you are listening to signal and not the noise.
• But answering your research question can be done in other ways
• Behavioral Observations• Open ended qualitative questions• Many more…
04/07/23 27
The End.
So was this presentation simply awesome?
a. Hell yeah!b. Definitely!c. Totally rocked!
04/07/23 28
Thanks!