Dr Gary Jones

Business Research Methods

Questionnaires

Dr Gary Jones

Today

• Review of last week

• Questionnaire design

• Starting your questionnaire

Dr Gary Jones

Research Question

• Thinking about reliability, which statement below refers to stability

Whether or not the scale or index is consistent.

If we administer a measure to a group and then re-administer it, there will be little variation over time with the results.

The extent to which your data collection techniques or analysis procedures will yield consistent findings.

Bryman & Bell, 2007

Dr Gary Jones

Research Question

• Thinking about validity, which statement refers to face validity

A measure covers all possible dimensions of the research topic.

Refers to the extent to which the research design impacts on the research outcomes.

That the measure apparently reflects the content of the concept in question. It measures what it is expected to measure.

Bryman & Bell, 2007 & Sarantakos, 2005

Dr Gary Jones

Criteria of validity and reliabilityValidity• Is a measure of the quality of

measurement• Tests the quality of indicators

and research instruments• Measures relevance, precision

and accuracy• Tests the ability to produce

findings that are in agreement with theoretical or conceptual values.

ASKS: Does the instrument measure what it is supposed to measure?

Reliability • Is a measure of the quality of

measurement• Tests the quality of indicators

and research instruments• Measures objectivity, stability,

consistency and precision• Tests consistency, i.e. the

ability to produce the same findings every time the procedure is repeated

ASKS:Does the instrument produce the same results every time it is employed?

Sarantakos, 2005: p. 91

Dr Gary Jones

• Types

PostalPostal

ElectronicElectronic

Self-completionSelf-completion

Dr Gary Jones

– Presentation – Instructions– Headings– Short and simple is usually best

Dr Gary Jones

– Cost– Speed– Absence of interviewer effect (postal)– Convenient– Anonymity – Cover a wide geographical area

Dr Gary Jones

– No opportunity to probe or prompt– Waste paper bin– Difficult to have complex questions– Who is answering the questionnaire?– Literacy– Missing data– Low response rate

Dr Gary Jones

Open Closed

• Advantages of open questions– Answer in own terms– May get responses that you have not

considered previously– Explore new areas– Help to generate fixed-choice responses

What are your thoughts about e-shopping?

Dr Gary Jones

• Personal factual questions– Age

• Factual questions about others– Productivity of team members

• Informant factual questions– Who owns your firm

• Attitudes– How strongly do you feel about global

warming?Bryman & Bell, 2007: p.264

Dr Gary Jones

• Beliefs– Do you believe in God?

• Normative standards and values– Is underage drinking acceptable? / What

principles influence them

• Knowledge– Questions on health and safety in the

workplace

Bryman & Bell, 2007: p.264

Dr Gary Jones

– Time consuming

– Coding responses

– Require greater effort from respondents

– Effect of different researchers

Dr Gary Jones

– Easy to process– Comparability– May help to clarify a point– Easy to complete– No difference in the recording of answers

Do you fly with Virgin Atlantic? Yes No (please circle your response)

Dr Gary Jones

– Loss of spontaneity– May not fit in with the responses Loss of

spontaneity– May not fit in with the responses– Interpretation of the question– Frustrating and stop completing it– Impersonal feel

Dr Gary Jones

• Relevant• Avoid

– Ambiguous – Leading– Double-barrelled– Long questions– Jargon – Stretching memory– Prestige bias

• Sensitive

Dr Gary Jones

• Simple and clear language

• Flows

• Using Yes and No responses excessively

Dr Gary Jones

– Do you buy DVDs regularly?

– Do you think that bankers receive excessive bonuses?

– Leading lecturers in the field of business believe that increasing the rate of VAT is an excellent idea, what are your thoughts on this issue?

– Do you think teenagers are eating too much and exercising too little?

– Do you sometimes feel alienated at work?

– What was your very first book?

– What is electromagnetism?

Double-barrelledDouble-barrelled

AmbiguousAmbiguous

LeadingLeading

JargonJargon

Prestige BiasPrestige BiasMemory problemsMemory problems

Requisite knowledgeRequisite knowledge

Dr Gary Jones

• What is your weekly rent? (please tick the appropriate box)– £40 - £50– £50 - £60– £60 - £70– £70 - £80– £80 - £90

Dr Gary Jones

• Do you visit Sainsbury’s regularly?– Once a week – Once a month – Once a year

Dr Gary Jones

– Go from easy to difficult– Go from general to particular– Go from factual to abstract questions– Put your questions under themes– Leave demographic and personal questions

until the end– Put yourself in the participant’s shoes

Dr Gary Jones

Household NeighbourhoodRepairs service Communication

with landlord

Questions

Questions

Tenant Satisfaction

Questions

Questions

Visit http://www.housing.org.uk/Uploads/File/Policy%20briefings/STATUS_4NA.pdf

Dr Gary Jones

• Tick all of the coffee shops that you have visited this year?

• Costa• Café Nero• Starbucks• BHS• M&S

• Which coffee shop do you visit most frequently?

• Costa• Café Nero• Starbucks• BHS• M&S

Dr Gary Jones

– The reaction of students to the recent tuition fee increase was in my opinion:

– Very positive– Positive– Neutral– Negative– Very negative

Dr Gary Jones

• Place in order of importance to you the following when choosing a Mp3 player? (1 very low and 4 very high)– Size– Brand– Price– Features

Dr Gary Jones

– How do you rate the following aspects of a plane flight (please circle)?

Very

Important

Important Neither Unimportant Very Unimportant

Seating space 1 2 3 4 5

Food 1 2 3 4 5

Service 1 2 3 4 5

Dr Gary Jones

– How much physical pain are you experiencing in your arm?

1

2

3

4

5

No pain

Unbearable

Dr Gary Jones

Please choose the face that appropriately describes your experience of using T-mobile over the last year– 1 delighted / 7 terrible (very happy / very unhappy)– Which face comes closest to expressing how you

feel about your life at the moment?

Dr Gary Jones

• Balance your question responses– What is wrong with this question:

• How satisfied / dissatisfied are you with your computer service?

– Very Satisfied / Satisfied / Neither / Dissatisfied

• What are your thoughts on the use of ‘don’t know’ in questionnaires?

Dr Gary Jones

• What is the value of a pilot study?– Testing your

questionnaire before sending it out to your participants

Dr Gary Jones

• High– 1/3

• No time, too busy

• Complexity of the questions

• Lost in the post

• Unable to complete the questionnaire– E.g., literacy

Dr Gary Jones

• Explain why it is important to obtain the views of your participants – try to make them feel special

• Not too long• Not to difficult• Very clear instructions• Confidentiality & anonymity• Allow time

Dr Gary Jones

• Pre-paid envelope

• Professional presentation

• Limited sensitive questions

• Send a reminder letter

• Incentives

Dr Gary Jones

Choose three colours.

1. What do you associate with these colours

2. Would a certain colour stop you from buying something or completing something

Dr Gary Jones

– Pressley and Tuller (1977) • White paper 43.7%, Yellow paper 38.8%, Green paper 37.3% &

Blue paper 35.8%• Marketing Directors

– Crittenden & Crittenden & Hawes (1985) • Yellow 49%, White 37%• Students

– Jobber & Sanderson (1983)• Blue highest response rate, but not significant• Textile managers

– Fox, Crask & Kim (1988)• Green 5% higher than white

– Diaz de Rada (2005)• Green paper

Source: Diaz de Rada, V. (2005) Influence of Questionnaire Design on Response to Mail Surveys. International Journal of Social Research Methodology, Vol. 8, No. 1, pp.61-78.