Effective Questionnaire Design
description
Transcript of Effective Questionnaire Design
Effective Questionnaire
Design
Craig Parylo & Annette Lee
What is a questionnaire?
A series of questions.
Gathers information
from lots of people.
Focussed around a
single topic or area of
interest.
An effective questionnaire…
Asks relevant questions• Questions that will provide the answers to the
research topic in question
Gets valid responses• Responses given reflect the respondent’s actual
opinions
Gets a representative response• Makes it more reliable to infer the findings to
wider population
Survey
report
Food
Question 1
Question 2
Question 3
Introduction
Anyone can design a questionnaire.An effective questionnaire requires planning and an appreciation of the psychology of respondents:Filling in a questionnaire is a complex process.• The job of a questionnaire designer is to
make it easy to respond.
Complex cognitive process
Do you drink coffee? Yes No
Food Question 1
1. Identify subject
2. Phrase as question
3. Reads
4. Decodes 5. Collects info
6. Share?
7. Encode
I prefer tea
8. Option No9. Respond
Exercise: responding to questions
1. What is the colour of your car?2. What do you usually have for breakfast?3. What did you have for breakfast last Friday?4. Have you ever not left a tip at a restaurant?5. What is your sexuality?6. What is the biggest regret in your life?
Choosing the right approach
Phot
o by
Ron
it G
elle
r (htt
p://
ww
w.s
xc.h
u/pr
ofile
/Rog
el)
Types of questionnaireSelf-
completed
Postal
Electronic
Interviewer-led
Face to face
Telephone
No method consistently outperforms others
Considerations
Your approach should take consideration of:• Your topic• Who you want survey• The type of information
you want• Your budget• How quickly you need
the information
Sample size vs Depth of info
• Better response rates from interviewer-led surveys than from self-response surveys.
• Self-response surveys are usually cheaper.
• Interviewer-led surveys can produce richer information.
• Self-response surveys can reach more people.
Don’t confuse quantity for quality.
Children
Is it pitched at the right age group?
Timing and convenience
• Will it be completed right after the event, or sometime after?
• Will it be completed in front of other people or in private?
• Many people are poor at estimating numerical info about their lives.
• The more common the event the shorter a person’s window of recall is.
Question types
Question types
• Open / closed• Single / multiple
response• Attitudinal / opinion
questions• Filtering / routing
Attitude/opinion
• Can be difficult to get the phrasing of the question right so it means the same thing to all people.
• Can be tempting to put lots of explanation into the question – but sometimes this just confuses things further.
• Use clear, simple English/ language and avoid being too general.
• Try not to ask ‘Iceberg’ questions.
Mid-point / neutral opinions
Use mid-points in questions on attitude.
• Historically controversial as it was thought that respondents chose a neutral option to avoid thinking about a question.
• Now shown that including a mid-point increases reliability and validity
Strongly disagree
Strongly agree
‘How important?’
Following your visit to the clinic today, please rate the following – first based on your experience and second how important this is to you?
1 2 3 4 5 6 7Waiting time Very poor Excellent
How important is the length of waiting time to you?
Not important Very
important
Quality of magazines in waiting room
Very poor Excellent
How important is the quality of magazines in waiting room to you?
Not important Very
important
17
Common mistakes
What might be wrong with this question?
What was the first drink you had
today?
Tea
Coffee
Water
Juice
Haven’t had a drink today
Other (please state)
Need to cover all options
Common mistakes
What might be wrong with this question?
What is your age group?0-18 18-30 30-45 45+
0-18 19-30 31-45 46+
Options should be mutually exclusive.
Common mistakes
What might be wrong with this question?
What did you think about the waiting time?Excellent Very
goodGood Average Poor
‘Good’ responses ‘Bad’ response
Options should be balanced
Common mistakes
What might be wrong with this question?
How would you rate the appointment booking service using the:
Good Average PoorWebsitePhoneReception
Use opt-out responses appropriately
N/A
Increasing clarity
22
Question wording
• Keep questions as short as possible– Max 16 words.
• Consider providing medium length (30 word) introductions to groups of related questions (context is vital!)
• Use open-ended questions sparingly
23
Double questions
If a question can be broken down into parts then it should be?
Is space and storage adequate?
☐ Yes☐ No
Was the consent of the patient
obtained and recorded?
☐ Yes☐ No
Was the consent of the patient:
☐ Obtained?
☐ Recorded?
Double negatives
• If the respondent has to tick ‘No’ to give a positive response then this can be considered a double negative.
• Try and phrase questions with a positive slant so that the positive response is ‘Yes’ rather than ‘No’.
25
Do you think that it is an unwise policy to continue sending food aid to Africa. ☐ Yes ☐ No
Style, appearance and layout
Design
• Use booklet format with double-sided printing– A4 folded to A5, or, A3 folded to A4
• Front cover:– The title of the questionnaire– Identity of the organisation carrying it out– Clearly explain the purpose of the questionnaire– Keep graphics neutral
Aesthetics
• Arial font, size 12 point or larger• Questionnaire length– Shorter is generally better– Also keep plenty of ‘whitespace’
• Be consistent in the use of colour, shapes and location in order to guide the respondent through the questionnaire
Other elements to remember
• Make it look professional.• Give an estimated completion time.• Put the deadline for return at the beginning
and end of the questionnaire.• Provide a return address – on the
questionnaire. • Say ‘Thank you’.
Ordering your questionnaire
• Place easy, non-challenging questions first• Key filter questions should go in early.• Use a funnel structure – Go from basic to more in depth.– General questions before specific questions.– Go from the least sensitive to the most sensitive
questions.– But – try and put important questions in before the
end.
Ordering your questions
• Ask for suggestions for improvement at the end.
• Check for leading questions.• Add prompts where required.• Order of questions can impact heavily on the
amount of time it takes for the respondent to complete the questionnaire.
Filtering
• Filtering (or routing) helps identify which respondents should answer a detailed set of questions about a certain aspect of the topic.
• Can be difficult to achieve – should be VERY CLEARLY signposted
Filtering ExampleAbout our new website Yes No1. Did you visit our website today?(If yes please answer questions 2-4, if no go to question 5)
2. Did you like the new look?3. Did you find it easy to navigate?4. Would you recommend the site to others?5. Did you pick up a copy of our latest leaflet?
✓
✓✓
✓✓
Filtering ExampleAbout our new website Yes No1. Did you visit our website today?
Please continue with Q1a
Please go to Q2
1a. Did you like the new look?1b. Did you find it easy to navigate?1c. Would you recommend the site to others?
2. Did you pick up a copy of our latest leaflet?
Tips for increasing response rate
Phot
os b
y Sa
rah
Will
iam
s & D
avid
e G
uglie
lmo
(http
://w
ww
.sxc
.hu/
profi
le/M
eHer
e) (h
ttp:/
/ww
w.s
xc.h
u/pr
ofile
/bro
kena
rts)
Improving the response rate (1)
• Shorter questionnaires• Good communication– Contact the respondents beforehand– Follow-up contact with non-responders
• Monetary incentives• Personal touch– Handwritten address– Including respondent’s name in cover letter
Improving the response rate (2)
• Method of delivery– Include a stamped return envelope– Send by first class or recorded delivery– University sponsorship can add credibility
• Assure confidentiality
Key take-home message
Pilot your tool
• This is the MOST IMPORTANT part.• If a range of respondents from different
backgrounds will be using the questionnaire make sure you use as many of these different groups as possible in the trial.
• Go back to your objectives – does the data help you fulfil your objectives and measure against the standards you set?
39