366_8. Survey Design Things to consider Layout Time to complete Types of questions Question...
-
Upload
hilary-wilkins -
Category
Documents
-
view
215 -
download
0
description
Transcript of 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question...
366_8
Survey Design
• Things to consider– Layout– Time to complete– Types of questions– Question wording– Sensitive items
– THE RESPONDENT
Survey Design
• THE RESPONDENT– Does she know what you are asking?– Does she understand the question?– Can she recall the needed information?
– Does she have a reason to hide something?
Survey Design
• THE RESPONDENT– May not be paying close attention
• Make survey easy to read, follow
• Use similar response categories for blocks of questions where possible
• Sensitive items at the end
Survey Design
• THE RESPONDENT– Needs to know why the survey is being done• Why bother to do it?
– Needs to know the responses are anonymous.• Phone vs..• Mail vs..• Internet • Handed to them (face-to face)
Types of Questions
• Open ended– What is the most important issue facing the
country today?
– Can you tell me what it is that makes you like the Democratic Party?
Types of Questions
• Open ended– Valuable to measure information
– Might be biased toward people with more information
Types of Questions
• Open ended– Very labor intensive to enter data
– Must create categories for responses, code into spreadsheet
Types of Questions
• Closed ended– Defined response categories
– Overall, how satisfied are you with how democracy is working in the United States• Very satisfied• Satisfied• Not very satisfied• Not satisfied
Types of Questions
• Closed ended– Do you want a neutral
category?• Agree strongly• Agree• Neither agree nor
disagree• Disagree• Disagree strongly
• Closed ended– Neutral category might
be easy response default• Too little• About right• Too much
• More• Same• Less
Types of Questions
• Closed ended– Are you creating an ordinal (or even interval)
measure?• 1) a very great deal 6) Very little• 2) a great deal 7) None• 3) quite a bit• 4) a fair amount• 5) a little
Types of Questions
• Closed ended– Thermometer scales• On a scale from 0 to 100, with 0 being cool, 50 being
neutral, and 100 being warm, tell me how you feel about [whatever]. Just write a number between 0 and 100 in the space below– [whatever1] _________– [whatever2]__________– [whatever3]__________
Types of Questions
• Closed ended– Forced choice?
• Should response categories include “don’t know”
• Phone vs.. print surveys– If phone, we might not offer DK, but code it
– If print, if no DK option, respondent might skip
Closed Ended Question Blocks
Question Blocks
Layout:
Question Wording
• Be specific– “are you liberal or conservative” vs.:
– When it comes to politics, do you consider yourself liberal or conservative?
Question Wording
• Avoid biasing responses– Leading questions• “Do you favor or oppose dirty coal?”
– Loaded words• “welfare” vs.. Income support for the less affluent
– Double – barreled questions• Should we raise taxes and cut spending to lower the
deficit?
Question Wording
• Avoid biasing responses
– Is Clinton an old-style, tax & spend liberal, or a new style Democrat who who will be careful with the nation’s money? [ABC] vs.:
– Is Clinton a liberal Democrat or a Conservative Democrat (or, see slide 14).
Question Wording
• Sensitive questions– If you must ask, ask indirectly
– Drinking, not voting, rioting
• Occasionally, people drink too much and become [intoxicated]. In the past year, how often have you become [intoxicated] while drinking any kind of alcoholic beverage?
Question Wording• Sensitive questions [Why didn’t you vote?]• In any election, some people are not able to vote because they are sick or busy or have some
other reason, and others do not want to vote. Which best describes your situation in the recent local election in [CITY]
• • Illness or disability (your own, or in your family)• Forgot• Registration problems• Not interested; it doesn't matter• Too busy• Disliked the candidates• Out of town• It was inconvenient
Don’t like the voting rules• Other (VOL)• Don't know (VOL)
More points about the layout
• Begin with brief intro– Class project, anonymity, etc.
• Follow with ‘easy’ substantive question– First we’d like to ask you about...
• Start questions casual, chatty, clear– In general, do you think_______– Thinking about _____ , – How about, _______, Do you think....
More points about the layout
• Most important substantive items next
• Qs that test your main hypotheses
• Demographic questions at end
• Check established surveys for coding– Age – Region of state– Education– Party– Ideology– Religion [ or / and ]– Religious attendance– Family income (last)
More points about the layout
• Whatever layout, include:
• Are you registered to vote?– If no, SKIP to Q X– If Yes, are you registered
to vote here in Whatcom County, or elsewhere?
Before you print
• Are you sure everything is in there?– You have what you need to test your main
hypotheses
– You have questions that measure the attitudes you are interested in
– You have the questions designed to explain variation in the attitudes you are interested in
More points about the layout
• Printing– Single sheet– Back to back
– Booklet
– Font size
Pre-test
• Once DRAFT is completed, give survey to a few friends– How long did it take?– Ask them about it after they take it
• Any items not make sense?
Monitor refusals
• Is there a pattern to who is not filling out survey?