1

Social Research Methods

Questionnaire Design

2

References

Oppenheim, A.N. 1978 Questionnaire Design and Attitude Measurement. London: Heinemann

de Vaus, D. A. 1996 Surveys in Social Research, 4th Ed. London: UCL Press (Chapters 6 & 15)

Foddy, W. (1993) Constructing Questions for Interviews and Questionnaires, Cambridge: Cambridge University Press.

Plus Hoinville, G., Jowell, R and associates (1985)

Survey Research Practice. London: Gower Moser, C. A. and Kalton, G (1971) Survey

Methods in Social Investigation. Aldershot: Gower

3

When is a questionnaire appropriate?

Have good idea of what to expect Literate respondentsor Time to administer if not literate Interested in counts, frequencies

etc. Want to make generalisations to

larger population

4

Procedure

Working out the aims of the questionnaire

Selecting the question styles Designing the questions Structuring the questionnaire Piloting the questionnaire Revising the questionnaire Administering the questionnaire.

5

Information which can be obtained from questionnaires

Facts and Knowledge (biography) Actions Opinions/beliefs Attitudes Motives Past Behaviour Likely Future Behaviour

6

Choosing the questions

What type of information do you need from this question?

How will the answers serve the aims of the questionnaire?

How will you present the findings from answers to this question?

Do you need numerical scores from the answers?

7

Open questions

1. What do you think the Government should be doing about global warming?

8

Closed questions

1. What do you think is the most important factor to consider when buying food? Tick just ONE box

a) Value for money 1

b) Hygienic processing 2

c) No chemical additives 3

d) Good flavour when cooked 4

e) Easy to prepare 5

9

Open or closed questions?

Open questions Closed Questions

More representative of respondent’s true

opinions

Very easy to score and analyse

Less open to researcher’s bias

Give predictable outcomes

Useful to indicate points of interest or

weaknesses

Useful for statistical illustrations or

summaries

10

Questionnaire structure

TitleIntroductory paragraphContent questions

in sectionsuse filter questionsput more difficult questions towards the

enddo not give away answers to later

questions

End and thanks

11

12

Questionnaire layout

Allow space Instructions (general, section, question,

filter question) Numbers for computer coding One side of page only (space for

comments)

13

Telephone Questionnaires

Retention problem

= too many categories in answer to remember

1. use numbered scale

2. get direction of feeling then intensity

3. include in question then repeat list

Provide guides for interviewer on questionnaire – distinguish from questions

Make clear what is to be read to respondent and what is not (e.g. instructions for prompt)

14

Checklists and ratings

15

Semantic Differentials

How would you rate your mother on these scales?

Warmhearted I I I I I Coldhearted

Patient I I I I I Impatient

Self-assuredI I I I I Always worried

Unselfish I I I I I Selfish

16

Why pilot a questionnaire?

1. To expose the questionnaire to new forms of thinking and understanding.

2. To evaluate the quality of the answers provided by the questionnaire.

3. To assess and revise the phrasing and content of the questions.

4. To explore issues of questionnaire structure and question sequence.

17

Use Piloting to check for:

Variation in type of answer Meaning Redundancy Scalability Non-response (too much) Acquiescent response (too much

agreement with agree/disagree Q) Flow Filtering Timing

18

Scaling

Needs to be Unidimensional Linear (equal intervals) Reliable Valid (esp. for standard scales)

19

Major types of scales

Thurstone Judges rank agree/disagree items

Lickert Judges score large list of items

Guttman/Scalogram Items eliminated to form a ranked scale

Paired comparisons Pairwise comparing of items

20

Lickert Scales

21

22

23

Some problems of question-writing:-

Leading questions

Are you against giving too much power to trades unions?

Why don’t you go more often to the supermarket?

24

Ambiguous questions

Is your work made more difficult because you are expecting a baby?

Also words like ‘Books’, ‘Dinner’

Multiple-content questions

Have you suffered from headaches or sickness lately?

25

Over-complex vocabulary

Avoid big words and technical terms

e.g. “Personality”, “Social mobility”, “Anomie”, “Patriarchy”.

Simple alternativesacquaint inform or tell

assist help

consider think

initiate begin; start

major important; chief; main;

purchase buy

require want; need

reside live

state say

sufficient enough

terminate end

26

Over-elaborate phrasing and too complex question

Has it happened to you that over a long period of time, when you neither practised abstinence, nor used birth control, you did not conceive?

Yes No

vague (what is a ‘long period’ too factual (‘has it happened to you that …’) complex - Long sentence and double

negative

27

Implicit questions (based on assumptions)

How many children do you have? (Asked of all people)

How old is your car?

28

How often?

Use either:

In the last week how often .…? On average …? When did you last .…?

29

Embarrassing questions

e.g. sexual mores, alcohol consumption, law breaking, taboo subjects (e.g. death, menstruation)

Create permissive atmosphere by range of possible answers.

e.g. not “Does your child steal?”

but,

“Which of the following best describes your child?

1.He or she never steals

2.He or she sometimes steals (including from home)

3.He or she has stolen on several occasions.

30

Vague terms

e.g.

a long time

recently

near you

kind of

fairly

often

on the whole

31

Prestige bias

“Doctors say that …” “Research has shown that …” “Is the government right to .…”

32

Patronising tone

Avoid this.

“Answer these questions to the best of your ability”

“‘Dwelling’ means the place where you regularly live”.

Use the Sun/Mirror style