Spring€2013 School€Administrator€Questionnaire Questionnaire€A
Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.
-
Upload
nancy-leslie-hutchinson -
Category
Documents
-
view
216 -
download
0
Transcript of Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.
![Page 1: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/1.jpg)
Survey and Questionnaire Design
MAR 6648: Marketing ResearchJanuary 20, 2010
![Page 2: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/2.jpg)
Overview
• How do you write survey questions?– What are some common traps survey writers fall
into?– How do we avoid them?
• What about special circumstances?– Embarrassing questions, etc.?
![Page 3: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/3.jpg)
Survey Research
• Descriptive research• Survey modes and methods:– Telephone surveys– Mail surveys– Face-to-face surveys• At home• Mall intercepts
– Web-based surveys
![Page 4: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/4.jpg)
Advantages of Surveys
• Can collect a great deal of data, including– Attitudes, interests, opinions, behaviors– Classification variables
• Can be employed in virtually any setting• Can be inexpensive, depending on format
![Page 5: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/5.jpg)
Disadvantages of Surveys
• Effective implementation requires considerable judgment
• Social interaction of interview may lead to errors
• Subject to demand and honesty concerns
![Page 6: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/6.jpg)
First Big Point: Biases
• Basic Tenet: Ask a question, and you’ll get a response!
• Translation: Garbage in, garbage out!
![Page 7: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/7.jpg)
Typical problems in wording questions
![Page 8: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/8.jpg)
How questions create answers
![Page 9: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/9.jpg)
How questions create answers
![Page 10: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/10.jpg)
How questions create answers• A survey for Black Flag said:
– “A roach disk… poisons a roach slowly. The dying roach returns to the nest and after it dies is eaten by other roaches. In turn, these roaches become poisoned and die. How effective do you think this type of product would be in killing roaches?”
– Not surprisingly, 79% said effective!• A Gallup poll sponsored by the disposable-diaper industry
asked:– “It is estimated that disposable diapers account for less than 2% of the
trash in today’s landfills. In contrast, beverage containers, third-class mail and yard waste to account for about 21% of the trash in landfills. Given this, in your opinion, would it be fair to ban disposable diapers?”
– 84% said no!
![Page 11: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/11.jpg)
How questions shape answers
• Some examples– Behavioral frequency questions• “How often do you…?”
![Page 12: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/12.jpg)
Behavioral Frequency Questions
• How many hours of TV do you watch on an average week day? Please choose the category from the list below that best describes your behavior.
Marketers Question: What % of class watches more than 2.5 hrs of TV?
![Page 13: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/13.jpg)
Behavioral Frequency Questions• Scale represents
distribution in ‘real’ world– Values in middle reflect
“average” / “usual” behaviors
– Values at an end (or ends) reflect extreme behavior
• Low Frequency Scale– Lower estimates of
“average”– Avoid marking extreme high
box• High Frequency Scale
– Makes marking “>2.5 hours” more acceptable
• “Response Scale Effect”
![Page 14: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/14.jpg)
How questions shape answers
• How much money do you have in savings?– 1-7; 1 = $0, 7 = $500 and
over– 1-7; 1 = $500 and under,
7 = $500,000 and over
• How wealthy are you?
![Page 15: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/15.jpg)
How questions shape answers
• Some examples– Behavioral frequency questions– Aided versus unaided questions
![Page 16: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/16.jpg)
Aided and Unaided Questions
• What do you consider to be the most important thing for children to prepare them for life?– 1. _______________– 2. _______________– 3. _______________– 4. _______________
• What do you consider to be the most important thing for children to prepare them for life?– __Good education– __Inheritance– __Moral values– __To think for themselves– __Good friendships– __Others (please specify)
![Page 17: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/17.jpg)
How questions shape answers
• Some examples– Behavioral frequency questions– Aided versus unaided questions– Framing of questions
![Page 18: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/18.jpg)
Framing Effects
• Question forms that lead respondents to give a certain response– Example: Allow versus forbid
Q: Do you think the U.S. should forbid public speeches against democracy?
Q: Do you think the U.S. should allow public speeches against democracy?
Yes No
5% 95%
81% 19%
![Page 19: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/19.jpg)
Framing Effects
• Question form• Choice of example/source
![Page 20: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/20.jpg)
Framing Effects
• Question form• Choice of example/source– Surveyor power: Mispredicting political outcomes
(Bischoping & Schuman 1992)– Perhaps partly motivational?• People who feel good about themselves show less of
this bias (Cohen 2003)
![Page 21: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/21.jpg)
How questions shape answers
• Some examples– Behavioral frequency questions– Aided versus unaided questions– Framing of questions– Scales associated with questions• Odd versus even numbered scales• Semantic versus Stapel scales
![Page 22: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/22.jpg)
How questions shape answers
• Some examples– Behavioral frequency questions– Aided versus unaided questions– Framing of questions– Scales associated with questions– Sequence of questions
![Page 23: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/23.jpg)
Sequence of Questions
• Specific – General Questions– Happiness and dating (How many dates have you had
recently?)• Order 1 (Happiness then dates): Correlation = 0.03• Order 2 (Dates then happiness): Correlation = 0.67
Q1: General Colin Powell, the hero of the Gulf War, recently decided to become a member of a political party. Do you happen to know which party that is?
Q2: Now rate this party.
Q1: General Colin Powell, the hero of the Gulf War, recently decided to become a member of a political party, but decided not to run for the presidential elections. Do you happen to know which party that is?
Q2: Now rate this party.
![Page 24: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/24.jpg)
Sequence of Questions
• Difficultly in answering one question can influence subsequent answers
• List three (twelve) ways in which you are happy (unhappy) with your partner.
• How happy are you with your partner?
![Page 25: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/25.jpg)
Sensitive Questions
• Respondents may not be willing to answer some questions, due to embarrassment, legal issues, etc.
• This may result in – Item nonresponse– Termination– Distortion
![Page 26: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/26.jpg)
Asking the Embarrassing Question• The casual approach
– “Do you happen to have killed your wife?”• The numbered card approach
– “Would you please read off the number on this card which corresponds to what became of your wife?:• 1: Natural Death• 2: I killed her• 3: Other
• The everybody approach (counter biasing method)– “As you know, many people have been killing their wives these
days. Do you happen to have killed yours?”• The other people approach
– “Do you know any people who have murdered their wives? How about yourself?”
Barton, POQ, 1958
![Page 27: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/27.jpg)
How to ask a sensitive question
• Personal information – Build rapport, legitimize yourself
• Embarrassing info – Downward bias– Counterbiasing statement• “Recent studies indicate that men often use their
wives’ hair spray. Do you use your wife’s hair spray?”
– Projective techniques
![Page 28: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/28.jpg)
Projective Techniques
![Page 29: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/29.jpg)
Key Points
• When wording questions…– Use simple words– Use clear words– Avoid leading questions– Avoid implicit alternatives– Avoid implicit assumptions– Avoid double-barreled questions– Consider frame of reference
![Page 30: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/30.jpg)
Key Points
• Don’t forget about question sequence– It can influence the nature of the respondent’s
answers and can be the cause of serious error in the results• Use a simple and interesting opening question• Ask general questions first• Place uninteresting and difficult questions late in the
sequence• Arrange questions in logical order
![Page 31: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/31.jpg)
The Basic Steps of Questionnaire Design
• Decide on question content• Decide how to ask each question– Response format– Question wording
• Decide on question sequence– Funnel approach (general to specific)
• Decide on physical characteristics• Carry out pretest, revise, and final draft
![Page 32: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/32.jpg)
So…
• What is a good survey question?• How are we to interpret survey question
output?
![Page 33: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/33.jpg)
A Managerial Problem
• Temptation: – Report those numbers to
your manager as is
• Problem:– Those numbers have no
direct value• What does “somewhat
likely” mean in real life?
In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely
is it that you will buy a Verizon iPhone?
In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely
is it that you will buy a Verizon iPhone?Very
UnlikelySomewhat
UnlikelySomewhat
LikelyVeryLikely
15% 35% 35% 15%
![Page 34: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/34.jpg)
In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely
is it that you will buy a Verizon iPhone?
In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely
is it that you will buy a Verizon iPhone?Very
UnlikelySomewhat
UnlikelySomewhat
LikelyVeryLikely
15% 35% 35% 15%
A Managerial Problem
• Solutions:– 1. Benchmark
• What do past surveys say?
AT&T iPhone 12% 28% 30% 30% 9%
VZ Droid Incredible 20% 25% 30% 25% 5%
Actual Purchase Rate
![Page 35: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/35.jpg)
A Managerial Problem
• Solutions:– 1. Benchmark
• What do past surveys say?
– 2. Focus on questions you can answer• Relative responses on the
same items
In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely
is it that you will buy a Verizon iPhone?
In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely
is it that you will buy a Verizon iPhone?Very
UnlikelySomewhat
UnlikelySomewhat
LikelyVeryLikely
15% 35% 35% 15%
Verizon Customers 35% 35% 20% 10%
AT&T customers 15% 25% 30% 30%
![Page 36: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/36.jpg)
A Managerial Problem
• Solutions:– 1. Benchmark
• What do past surveys say?
– 2. Focus on questions you can answer• Relative responses on the
same items
– 3. Ask other questions/use other methods
In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely
is it that you will buy a Verizon iPhone?
In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely
is it that you will buy a Verizon iPhone?Very
UnlikelySomewhat
UnlikelySomewhat
LikelyVeryLikely
15% 35% 35% 15%
![Page 37: Survey and Questionnaire Design MAR 6648: Marketing Research January 20, 2010.](https://reader030.fdocuments.in/reader030/viewer/2022032709/56649ed35503460f94be3846/html5/thumbnails/37.jpg)
Summary
• Survey design is harder than it looks– Be sure to think about what the question suggests
about the answer• Order and scale points matter, too
– Be sure to keep respondents’ comfort in mind