1

ALC208Week 8-Topic 7

Survey Research

Assigned Readings:

Text: Chapter 8; Reading 7.1: Moody (2004) & Reading 7.2: Weerakkody (2004): Reading 7.3: US Census (emailed)

2

Surveys Commonest data collection method in quantitative

research Census every five years in Australia, 10 yrs in the

USA Self-administered (given to respondents to

complete) or Researcher –administered (face2face or over the

phone ) Survey interviews – structured; Depth (semi-

structured), intensive (unstructured) interviews in field studies

Survey questions in: MCQ exams & SETUs Closed ended questions give quantitative data

and open ended questions give qualitative data

3

Survey QuestionsOperationalise variables with structured

questions an relevant categories of responses to a (close ended) question

e.g. what is your sex? 1. Male 2 FemaleAge: measured as nominal, ordinal,

interval or ratio variablesSurveys can use constructs to measure

concepts or variables that cannot be directly observed or measured using Likert or Semantic differential scales.

4

What surveys can and cannot do Checks factual knowledge of respondents (e,g.

MCQs at exams). Collects info on people’s beliefs, attitudes,

perceptions , opinions etc. (e.g. Yes/No; True/False; Agree/Disagree; their feelings (e.g. Positive/negative, like/dislike); behaviours (e.g. On a scale of 1 (strongly disagree) to 7 (strongly agree)

Can compare and describe relationships between variables. e.g. Sex, party affiliation and support for a policy

Tells about correlations between variables – not their causality (as one causes the other/s)

Cause and effect relationships need experimental designs where variables can be controlled and examined.

5

Survey MethodologyThe steps involved:1. Developing the survey

instrument / questionnaire2. Selecting the population and

sample (of respondents)3. Administration of the survey4. Data analysis

6

1. Developing the survey instrument / questionnaireDecide on research questions /hypotheses

to be tested in study and variables to be examined

Phrase and organise questionnaire Decide if to be self-administered or

researcher-administeredDecide if to be face-to-face, phone, mail,

email or internet basedHow much time needed to complete the

surveyWhat closed and open ended to be included

7

2. Selecting the population and sample (of respondents)Decide what the target population is.(e.g. Registered voters in Australia)Obtain the sampling frame (e.g. voter

registration lists)Choose a suitable sampling method (e.g.

a convenience sample for an exploratory study or a random, representative sample if generalisation of findings is needed)

Weighting (higher proportions of specific groups such as the older, females, majority ethnic group, uni students etc.)

Balancing (in proportion to census figures)

8

3. Administration of the surveyFace-to-face interviews

Telephone interviews

Mail surveys

Online (email or internet) surveys

SurveyMonkey, MySpace

9

Raliability & Validity of SurveysImportant to consider when

designing a surveyInterviewers need to be well-

trained and survey instrument tested with a few people from the same population.

Several other factors affect reliability (give same results when repeated) and validity (measure what you are supposed to measure).

10

Factors Affecting Reliability and Validity1. Unreliability of answers given by

respondents; they forget things etc. – Joke answers or missing data

2. The Social desirability effect

3. Unstable opinions of respondents

4. Question wording- e.g. Push polling

5. Misinterpretation of questions by respondents

11

Factors Affecting Reliability and Validity (Contd.)

6. Question order : Funnel format & reverse funnel format

7. Response set eg. The ‘Donkey Vote’

8. Question format – closed or open ended

9. Unqualified respondents

10. Cultural bias of questions. Eg. Big Brother, Caviar

12

Secondary Analysis of DataUsing data collected in previous

studies in a new study. e.g. census data

Uses data from archives for a fee Disadvantage of problems in

previous study passed on to new one

But the national census and other well designed studies provide inexpensive source of good data

13

Things to Remember1. Ensure mutually exclusive

categories of responses2. Responses must match question

posed3. Responses in logical order in a

continuum4. Question clarity5. Avoid double-barrelled questions6. Respondent competent to answer

14

Things to Remember (Contd.)7. Respondents’ willingness to answer- to

reduce no responses and social desirability effect

8. Simplify the questions and make them readable

9. Avoid negative wording ‘We should not do X’ etc. to be avoided

10.Avoid biased wording (e.g. A supreme court decision on X says...; and use neutral terms: Not induce perceptions such as attractiveness of a person related to the survey question

15

How to Organise a Questionnaire Question order- follow a coherent order

Contingency ordering- Question’s relevance to those that follow.

E.g.1. ‘Do you have email at home?’ Yes / No This is a filter question.

If no, go to question 5 etc. as questions 2 to 4 are about home email use irrelevant to those without email.

Include demographic questions

Layout & design

Giving instructions

Pre-testing

16

Analysis of Survey DataClose ended questions provide

uniform answers to surveys questions- easy to analyse, quantitative data.

Use of Excel software of manual coding using /, //, ///, ////, //// to indicate 1 to 5 times

SPSS- Statistical Packages for the Social Sciences- Version 17

Open ended questions give qualitative data to be analysed using common themes

17

Strengths and Limitations of Survey researchStrengths: can use large samples

when self-administered to describe characteristics of large populations. Flexible and allows for many questions to be asked at once.

Limitations: Subject to artificiality, Only collects self-reports of past and future or hypothetical action

Some topics not suitable for surveys

18

Use of Surveys in Journalism, PR and Media and CommunicationJournalism- Journalists calling people

or using other people’s surveys when news gathering ; About the profession with working journalists

PR- program evaluationMedia and Communication-

Adoption and use of a new media technology

e. g. The 2006 Aus. census asked about internet access at home.

19

Any questions?