SURVEY RESEARCH

• Babbie (2011, p.269): “Surveys are a very old research technique. In the Old Testament, for example, we find the following:”

• After the plague the Lord said to Moses and to El-eazar the son of Aaron, the priest, “Take a census of all the congregation of the people of Isreal, from twenty old and upward.” (Numbers 26:1-2).

Types of Surveys

• Telephone

• Face-to-face

• Mail

• Internet (web/email)

• Administered

• Self-administered

Key Concerns/Issues

• Sampling issues

• Response rate

• Clarity of concepts/constructs/questions

• Non-attitudes (door-stop answers)

• social desirability bias

Guidelines for Asking Questions

• Respondents must be willing to answer.

• Questions should be relevant.

• Short items are best.

• Avoid negative items.

• Avoid biased items and terms.

Guidelines for Questionnaire Construction

• Be aware of issues with ordering items.

• Include instructions for the questionnaire.

• Pretest all or part of the questionnaire.

Question/Response Wording

• Likert Scales

• Thermometer ratings

• Agree/Disagree statements.

• Ordered responses.

• Open responses.

• Check all that apply.

– open-ended: allows subjects more leeway, flexibility

– “What is your primary ethnic/cultural background?”

– requires a content analysis of responses

– close-ended ties respondents’ hands somewhat

– easy to code the data

– always include an “other_____________” category

Phrasing questions• Avoid evaluative language

– Phrasing of questions should not imply approval, disapproval

– Follow-ups should not suggest surprise, liking, disliking, etc.

• Bad example: Do you think the Democratic dominated Congress should lift the harsh restrictions on stem cell research?

• Bad example: Despite its poor track record in crisis intervention, do you think the United Nations should intervene in Darfur, Sudan?

Acceptable Response Rates

• 50% - adequate for analysis and reporting

• 60% - good

• 70% - very good

• This is a rule of thumb that may change depending on time period, method, and the book you are looking at

10

Other Issues

• Pre-testing• Follow-up mailings or telephone calls• Training of interviewers• CATI• Legal issues

– Minors– Autodialing of cell phone numbers

Guidelines for Survey Interviewing

• Dress in a similar manner to the people who will be interviewed.

• Study and become familiar with the questionnaire.

• Follow question wording exactly.• Record responses exactly.• Probe for responses when necessary.

additional considerations

• gender of researcher/respondent can influence responses

• ethnicity of researcher/respondent can influence responses

• medium can influence responses– face-to-face interview, telephone survey, mail

survey– anonymous versus non-anonymous

questionnaire• fatigue: minimize length of surveys

Strengths of Survey Research

• Useful in describing the characteristics of a large population.

• Make large samples feasible.

• Flexible - many questions can be asked on a given topic.

Weaknesses of Survey Research

• Can seldom deal with the context of social life.

• Inflexible in some ways.

• Subject to artificiality.

• Weak on validity.

Survey Problems

• Literary Digest presidential survey of 1936; post cards. Over 2 million respondents.

• Predicted Alf Landon over FDR• Survey predicted the outcomes of the

1916, 1920, 1924, 1928, and 1932• What went wrong?

– Based on telephone and automobile registration

– Response rate

Survey Problems

• Hite Sexuality Survey – 70% of women married 5 years or more are having sex outside of marriage (4,500 completed surveys out of 100,000).

• Recorded phone surveys.

• Push & Entertainment polling: political telemarketing masquerading as a poll

Bush campaign in South Carolina asked the following:

"John McCain calls the campaign finance system corrupt, but as chairman of the Senate Commerce Committee, he raises money and travels on the private jets of corporations with legislative proposals before his committee. In view of this, are you much more likely to vote for him, somewhat more likely to vote for him, somewhat more likely to vote against him or much more likely to vote against him?"

Video

Political Polling C-SPAN Video

http://www.c-spanvideo.org/program/54099-1