SURVEY RESEARCH

BUSN 364 – Week 12Özge Can

Survey Research

Asks a large number of people (respondents) the same questions about their beliefs, opinions, characteristics and past and present behaviors Standard questionnaire forms Self-reported beliefs and behaviors Produce information that is statistical in nature

=> quantitative Asks many questions at once, thereby measuring

many variables => one can test multiple hypotheses

Alternative explanations are statistically controlled

What Is Asked in a Survey?

Behavior How frequently do you brush your

teeth? Attitudes/beliefs/opinions

What is the biggest problem facing the youth today?

Characteristics What is your age? Are you married or

not?

What Is Asked in a Survey?

Expectations Do you plan to buy a car in the next

12 months? Self-Classifications

Which social class would you put your family into?

Knowledge Who was elected as the city mayor in

the last election?

Steps in Conducting a Survey (1):

Steps in Conducting a Survey (2):

Principles of Good Question Writing Two key principles guide writing good survey

questions: Avoid any possible confusion Keep the respondent’s perspective in

mind

We want a survey to provide a valid and reliable measure: Respondents should easily understand the

question’s meaning as you intended (clear, relevant and meaningful) and answer completely and honestly

Principles of Good Question Writing1. Avoid jargon, slang, abbreviations or

acronyms When surveying general public, use the language of

popular culture

2. Avoid ambiguity, confusion and vagueness “What is your income?” “Do you jog regularly?”

3. Avoid emotional language and prestige bias

“What do you think about paying murderous terrorists who threaten to steal the freedom of peace-loving people?”

Principles of Good Question Writing

4. Avoid double-barreled questions “Does your employer offer pension and health

insurance benefits?”

5. Avoid leading questions “You don’t smoke, do you?”

6. Avoid questions beyond respondents’ capabilities

“How did you feel about your brother when you were 6 years-old?”

“How much was your electricty bill last year?”

Principles of Good Question Writing7. Avoid asking about distant future

intentions “Suppose 8-years later you become a manager, how

would you behave to your employees?”

8. Avoid double negatives “I ain’t got no job” “Do you agree or disagree that students should not

be required to take a comprehensive exam to graduate?”

9. Avoid overlapping or unbalanced response categories

Make response choices mutually exclusive, exhaustive and balanced.

Getting Honest Answers

Sensitive Topics => Illness and disability (mental health problems, cancer), illegal or deviant behavior (drug or alcohol use, law violations), financial status (income, debts, occupation) Threatens people’s presentation of themselves;

their positive image. We should particularly cautious about these questions and the results. To increase honest answering:

1. Create comfort and trust2. Use enhanced phrasing3. Establish a desensitizing context4. Use anonymous questioning methods

Getting Honest Answers

Social Desirability Bias => occurs when respondents distort answers to conform to popular social norms. They give a socially acceptable answer rather than an honest answer Ex: people tend to overstate being highly cultured,

being outgoing and fun, giving money to charity, having a good marriage, loving their children

One way to reduce it: phrase questions in ways that make norm violation appear less objectifiable and give respondents “face-saving” alternatives

Open-Ended vs. Close-Ended Questions

I. Open-ended question: A type of survey inquiry that allows

respondents freedom to offer any answer they wish to the question

II. Close-ended question: A type of survey inquiry in which respondents

must choose from a fixed set of answers

Partially open questions => a set of fixed choices with a final open choice of “other”

Open-Ended Questions

Open-Ended Questions

Close-Ended Questions

Close-Ended Questions

Valid Responses

Three types of answers from respondents yield invalid responses: Swayed opinion => falsely overstating a position False positive => selecting an attitude position

but lacking any knowledge on the issue False negative => respondent refuses to answer

the question when he/she actually has information or an opinion

Neutral positions: Should we offer respondents who lack knowledge or have no position a neutral or “no opinion” choice?

Valid Responses

Response Set (Reponse Bias) => tendency of respondents to agree with every question in a series rather than carefully thinking through one’s answer to each

To avoid it: include different question types; change the sequence of response categories

Response Set Bias:

Valid Responses

To get valid responses: Choose the appropriate response style for the

question: agreement scales, rankings or ratings?

Present the alternatives fairly Attach numbers to a response scale to assist

respondents and give them a clue for understanding

Use visual presentations, colors, symbols and pictures

Appropriate question format and questionnaire design

Valid Responses

Questionnaire Design Issues

Length of survey or questionnaire No absolute proper length; it depends on the survey

format (e.g. mail, internet) and respondent characteristics

Short questionnaires are appropriate for the general population

Question order and sequence You should sequence questions to minimize

respondent discomfort and confusion After an introduction explaining the survey, make

opening questions easy to answer and pleasent. Order effect => answers to earlier questions can

influence later ones

Questionnaire Design Issues

Layout and format Questionnaire layout => Appearence of the

questionnaire form should be clear, neat and easy to follow; sheets, fonts, envelope, cover letter...

Question format => Should respondents circle responses, check boxes, fill in dots or write in a blank?

Nonresponse The failure to get a valid response from every

sampled respondent = nonresponse bias Can be a major problem if a high proportion of the

sample does not respond; lowers generalizability

Ways to Increase Response Rate

• Address to specific individual• Include a cover letter

– Request cooperation– Guarantee confidentiality– Explain purpose– Provide researcher contact information

• Include postage-paid, addressed return envelope

• Easy to follow questionnaire• Send follow-up reminders• Avoid conducting study during busy holidays• Back page for general comments• Advertise legitimate sponsors (university, gov’t

agency, etc.)• Small monetary incentives

Types of Surveys

Mail surveys Telephone surveys Face-to-face surveys Internet/web surveys

Types of Surveys

Features to consider when choosing a particular survey type: Administrative issues => response rate,

cost, speed, length Researcher control on questions =>

explaining the questions, flexible formats Success with different questions =>

open-ended questions, complex or sensitive questions

Sources of bias => social desirability, interviwer bias, respondent’s reading skills

Constructing a Survey – Some Resources

Many links for carrying out survey research:

http://managementhelp.org/businessresearch/surveys.htm

Information on designing surveys:http://www.sciencebuddies.org/mentoring/project_id

eas/Soc_survey.shtml

Online survey websites: SurveyMonkey (www.surveymonkey.com) PsychSurveys (www.psychsurveys.org)