Survey Design 101
description
Transcript of Survey Design 101
SURVEY DESIGN 101Gay Hylton
Institutional Research and Planning
WHAT YOU’RE TRYING TO FIND OUTOften there is the temptation to skip on preparation in order to move to the field too rapidly. This temptation should be avoided.
Ghislaine Delaine“The Social Dimensions of
Adjustment Integrated Survey”
INFORMATION WANTED: What information needs to be obtained to
meet the objectives of the survey Strategize with stakeholders Questions that need to be answered Focus groups Has the information been gathered in the past What are the outcomes desired Will basic demographical data be required/confidential How will the information be obtained?
Electronic survey Mail Telephone Interview Personal Interview Paper
INFORMATION WANTED: CONTINUED Determine what correlations of the data you
want to see. What will the answers to the questions allow you to
determine If X then Y
WHO YOU ASKOne common misconception is that the adequacy of a sample depends heavily on the fraction of the population included in a sample.
Floyd J. FowlerSurvey Research Methods
CHOOSING THE POPULATION All or representative sample http://www.surveysystem.com/sscalc.htm Is a comparative sample needed Response rate
# of responses/ # surveyed If survey needs to reflect the entire student
body population of HSU, complete a survey request form at http://www.humboldt.edu/irp/survey.html
HOW YOU ASK THE QUESTION“The goal is to have differences in answers reflect differences in where people stand on the issues, rather than differences on their interpretations of the questions.”
Floyd Fowler,Improving Survey Questionnaires:
Design and Evaluation
BASICS Be aware of your own biases Use language that survey participants will
understand Develop neutral questions Ask enough to cover topic adequately while
keeping the survey as short as possible Pay attention to the order of questions Provide exhaustive range of response
categories Write clear unbiased instructions
BASICS - CONTINUED Order
Matters Start with easy - proceed to complex Keep respondents interested Vary the question type
Simple is better Avoid technical jargons or concepts Use the same definitions and scales throughout the
form Watch for “double-barreled” questions Be specific
TYPES OF QUESTIONS Multiple Choice
Allow multiple responses? Ranking Scale Open Ended
SPECIFICS Multiple Choice
Lists Should be exhaustive while not being too long Categories should be mutually exclusive Allow respondents to provide multiple answers when
relevant When appropriate – use Other
Scales Odd or Even Likert or Numeric Provide clear, concise instructions on scale meaning Order matters – Positive to Negative vs Negative to
Positive Use the same scale throughout the survey
SPECIFICS - CONTINUED Open-Ended
Allows for spontaneous responses Use when you don’t know the answer Analyzing the responses can be difficult and time-
consuming
Question Logic Will all participants answer all questions?
HOW AND WHEN TO ASK“The questionnaire is only one element of a well-done survey.”
Don A. DillmanMail and Internet Surveys
– The Tailored Design Method
BEFORE YOU SURVEY Design your communication to participants
Personal or authoritarian Confidentiality? Explain the nature and reasons for the survey How it benefits them Include timeframe to answer survey
BEFORE YOU SURVEY - CONTINUED TEST
Pilot the survey with colleagues or a small sample of the population
Include all communications they will receive with the survey instrument
Helps ensures the reliability and the validity of your survey
For electronic surveys check for the flow of logic questions
TRAIN (If survey is to be administered via telephone or personal interviewer)
All interviewers need to conduct the survey in the same manner, using the same language
Design response sheets that include instructions to interviewers
INCREASING RESPONSE RATES“Sending questionnaires out is one thing; getting them back is quite another.”
Bill GillhamDeveloping a Questionnaire
TIMING MATTERS What are your target population’s habits Ask again but don’t ask too often Choose a different time to ask for each
additional request
COMMUNICATION Say please and thank you for your help Show positive regard – provide the reasons
for the survey and what the results will achieve
Asking for advice – “We need your input on this matter.”
Subject line of email – does it look like spam Does it need to come from an authoritative
source or a personal approach
TOOLSAll tools have intrinsic politics and technology is the tool of now.
Godfrey Reggio
FREE OR NOT Software
Survey Monkey Zoomerang Surveygizmo And many more
Design help Library – reference section “Survey Kit” , ebooks and
books on survey design IRP website has some links to reference material
QUESTIONS?“Who questions much, shall learn much, and retain much.”
Francis Bacon