SURVEYS AND QUESTIONNAIRES

See Robson Chapter 8

Typical Survey Features

Use of a fixed quantitative design Collection of a small amount of data in a

standard form from a large number Selection of representative samples from

known populations Generally closed-ended Not hard and fast rules

Survey Types

In-Person Questionnaire Self-Administered Postal (mail)

Questionnaire Telephone Survey Not restricted to the use of

questionnaires E.g. traffic survey – observational

Advantages of Surveys

All Survey Types: Simple and straightforward approach Adaptable to collect generalizeable

information from almost any population Data standardization

Interview Surveys: Interviewer can clarify questions Presence of interviewer encourages

participation

Advantages Cont.

Self-Administered/Mail: Way of retrieving information about history

of a large group Efficient – Large data sets, low cost, short

timeframe Note: May be disadvantage if researcher is

seduced into using a survey when it is not most appropriate to research question

Allows anonymity/encourages frankness Allows large geographical distribution

Disadvantages of Surveys

All Survey Types: Context, discourse, and meaning Lack of relation between attitude and behavior Social desirability response bias Data affected by characteristics of

respondents Self-Administered/Mail:

Low response rate – still representative? Misunderstandings not detected No control Respondents may not take seriously

Disadvantages Cont.

Interview Surveys: Data affected by interviewers/interview

bias Affected by interactions of

interviewer/respondent characteristics Concerns of anonymity – respondent less

open

Developing Survey Questions

Pilot work – semi-structured interviews, focus groups, other data collection

Previous studies Theoretical frameworks

Activities in a Sample Survey

1. Initial design and planning2. Designing the questionnaire3. Pre-Testing4. Final design and planning5. Data collection6. Analysis and reporting

1. Initial Design and Planning Most straightforward task for survey are ‘how

many,’ ‘how much,’ ‘who,’ ‘where,’ and ‘when’

Gauge public opinion Test theories Theoretical framework important when trying

to move beyond description to explanation Unit of analysis Population Sampling frame

2. Designing the Questionnaire Provides a valid measure of the research

questions Gets the cooperation of respondents Elicits accurate information Be clear Fixed-alternate responses should be

accurate, exhaustive, mutually exclusive, and on a single dimension

Keep response rate in mind for mail surveys Standardize in-person surveys

3. Pre-Testing

First stage: Informal – Colleagues, friends, family Ask if questions are clear, simple, and

unambiguous Second stage:

Respondents from the groups of interest Give thoughts on the question

Third Stage Formal Pre-test Pilot version

4. Final Design and Planning

Edit questionnaire Spelling Layout

Finalize coding and analysis procedures

5. Data Collection

Follow your plan! Keep clear notes/records Address issues as they arise

6. Analysis

Generally straightforward with closed questions

Codes can be arbitrary (but consistent) or can be the actual number

Have a code for non-response Open questions – Simplify many responses by

classifying into smaller number of groups Coding categories chosen from sample of all

respondents Coding effectively changes open questions to

defined set of responses

Survey Sampling

Selection from the ‘population’ Sample size

Would a smaller set of longer more detailed surveys be preferable to more?

Probability/representative samples Non-Probability samples

INTERVIEWS

See Robson Chapter 9

Interview Features

Generally one-on-one and face-to-face Can be in group settings or by phone Can be primary/only approach or in

combination with other methods Open-ended questions are the norm

Types of Interviews

Structured Extreme example is the survey

Semi-Structured Unstructured

Allows more flexibility of response Extreme is the ‘depth interview’

Respondent interviews/informant interviews

Advantages of Interviews

Flexible and adaptable Asking people directly is a short-cut in

seeking answers (versus observation) Offers possibility of follow-up Non-verbal cues Rich and highly illuminating material

Disadvantages of Interviews

Requires considerable skill/experience Lack of standardization/reliability

concerns/bias Time consuming and requires lots of

preparation Can be limiting geographically

Interview Process

Listen more than you speak Questions should be straightforward,

clear, non-threatening Avoid leading Enjoy it! Take a full record General format: Intro, Warm-up, Main

Body, Cool-Off, Closure

Interview Content

Content consists of: A set of items/questions

Closed Open Scale

Probes and prompts Proposed sequence of questions

Other Interview Types

Informant interviews Non-directive interview: Totally in control of

interviewee Focused interview:

Situational analysis Interview guide

Group Interviews Most common are a hybrid of discussion

and interview Allows group interaction Focus Groups

Focus Group Advantages

Highly efficient Checks and balances Focus on most important topics Enjoyable Inexpensive, flexible, quick to set up Stimulated by thoughts of others Contributions encouraged Those with limited reading/writing skills not

excluded Less inhibited members serve to break the ice

Focus Group Disadvantages

Limited number of questions Facilitating requires considerable expertise Needs to be well managed Conflicts may arise Lack of confidentiality May not be generalizeable Live and immediate Participants may try to impress one another Difficult to follow-up on views of individuals

Focus Groups

Homogeneous Groups: Facilitates communication Promotes exchange of ideas and experiences Gives a sense of safety May result in ‘groupthink’

Heterogeneous Groups: Can stimulate and enrich discussion May inspire new ideas Risk of power imbalances Can lead to lack of respect for opinions Can lead to dominant participant destroying group

process

Analysis of Interview Data

Taping and Transcribing Allows concentration on the interview Alternative to complete transcription is to

be selective Too late to think about analysis when

interviews are complete