Designing and Analyzing Questionnaires

Designing and Designing and AnalyzingAnalyzing

QuestionnairesQuestionnairesDr. GeneDr. Gene

Education InstituteEducation Institute

Purpose of QuestionnairesQuestionnaires are the most common data collection methods used in research. They can be used to assess:

– Attitudes– Opinions– Interests– Values

Advantages and Disadvantages

Advantages– Can assess a large group quickly– Easy to analyze if constructed correctly

Disadvantages– Requires “good” language skills– Some people give answers they think you want– Not very good for getting in-depth information

Types of QuestionnairesStructured

– Respondents select a response from those given

Unstructured– Respondents create a response

Combination– Contains structured and unstructured

items

Sample StructuredQuestionnaire

Why did you choose teaching as a career? Tick each reason that applies to you.

___ I like working with children.

___ The pay is better than most jobs.

___ I can help my country and society.

___ It is a safe profession for women.

Sample UnstructuredQuestionnaire

Why did you choose teaching as a career? In the space below give the reasons why you chose to become a teacher.

Types of Questions• Closed questions

Choices are provided to the respondents

• Open questionsRespondents must create responses

• Demographic questionsQuestions about the characteristics of the respondents

Types of Closed Questions

• Dichotomous Questions (2 choices)– Yes/No– Agree/Disagree

Do you think a paper recycling program should be started in your school?Yes No

Types of Closed Questions• Multiple-choice Questions

– Give respondents options– May ask for single or multiple answers

Example:How did you hear about our Website?

___ Newspaper___ Magazine___ Radio___ Internet___ Other: Please specify __________

Types of Closed Questions• Rank Order Questions

– Respondents place things in order

Example:Which activities do you like to do in your spare time? Place a “1” next to the activity that you like to do most, a “2” by the next favorite, and so on to the least favorite.

___ Watch TV___ Read___ Visit friends___ Surf the Internet___ Shop

Types of Closed Questions

• Rating Scale (Semantic Differential)– Also called a “Likert Scale”– Give a statement; choose your response

along a scale

Example:My students are motivated to learn.

StronglyAgree

AgreeNot SureDisagreeStrongly Disagree

Open Questions• Used to explore topics in-depth• Gives people a chance to respond in detail• They are time-consuming to summarize and

analyze

Types of Open Questions• Open-ended Questions

Example:What changes would you like to see in our science class?

• Stem Plus Questions

Example:Some things I would like to see change

in our science class are _________________

Demographic Questions• Demographic questions may ask

about personal characteristics such as --– age– grade– ethnicity– gender– home characteristics

Types of Scales• 2-level (yes/no; agree/disagree)• 3-level (yes/no/not sure)• 5-level (Likert scale – Strongly Agree,

Agree, Undecided, Disagree, Strongly Disagree)

• Larger scales are difficult to analyze

Effects of ScalesThe police need to do a better job of enforcing traffic laws.

AgreeDisagree90%10%

AgreeNeither Agree or Disagree

Disagree

70%28%2%

2-level

3-level

AgreeDisagree90%10%2-level

Effects of ScalesThe police need to do a better job of enforcing traffic laws.

Strongly Agree

AgreeUndecided

DisagreeStrongly Disagree

7%63%28%1%1%

5-level

Problems to Avoid in Writing Questions

Leading Questions (Loaded Questions or Biased Questions)Examples:Don’t you agree that discipline is a major problem in this school?

Do you believe it is all right to spank young children despite the SEC recommendations not to do so?


Double-Barreled Questions (asking two things in one question)Examples:Did this class increase your interest in science and motivate you to study hard?

Do you agree that pay is low for teachers and that the government should be working hard to increase it?


Unclear or ambiguous questionsExamples:What do you think about school?

What role should the principal play in educating students?


Halo Effect (questions that link a position to a particular person or group)Examples:Do you agree with the Emir that research should be one of the top priorities of Qatar?

Qatar Petroleum wants the Independent Schools to do a better job of teaching science so that students are prepared for the workplace. Would you agree?


Invasive questions about personal informationExamples:Do you ever argue with your parents?

What are the major problems in your home?

Tips for Creating Good Questionnaires

Know your audience

• Make sure that the length, content, and wording matches the intended audience

• Keep questions clear and concise• Avoid technical wording


Be certain of your goals

• Before you start writing questions, be clear in your own mind what you want to find out


Make the directions and format clear and easy to follow

•Put your directions and demographic questions at the beginning of the questionnaire.

•If possible, read and explain the directions orally


Try the questionnaire out before you use it

•Find a similar group that is not in your study.

•Ask them to read the questionnaire and tell you if something is not clear.

•Revise your questions based on the information you get from your tryouts.


Don’t ask too many questions•Keep your questionnaire short and to the point

•People get tired and lose interest on long questionnaires so the responses you get toward the end may not be valid.

Ways to Deliver Questionnaires

• Paper-pencil/face-to-face• Mail or take home• Telephone• Online delivery and analysis

Ways to Analyze The Results of a Structure QuestionnaireSummarize the percentages for each question

Students should be required to wear uniforms to school.

Strongly Agree

AgreeUndecided

DisagreeStrongly Disagree

20%40%10%20%10%

Ways to Analyze The Results of a Structure QuestionnaireCalculate the mean (average) for each item.Students should be required to wear uniforms to school.

Strongly Agree

(5)

Agree(4)

Undecided(3)

Disagree(2)

Strongly Disagree

(1)2040102010

Ways to Analyze The Results of a Structured Questionnaire

Calculate the mean (average) for each item.20 x 5 = 100 340 / 100 = 3.440 x 4 = 16010 x 3 = 3020 x 2 = 2010 x 1 = 10

340

Strongly StronglyDiagree Disagree 5 4 3 2 1

3.4

Ways to Analyze The Results of an Unstructured Questionnaire

• Summarize the comments• Look for patterns or themes that run

through the comments• Give percentages for major themes

Designing and Analyzing Questionnaires

Documents

Transcript of Designing and Analyzing Questionnaires