1

Exam 1 Review

2

COMM 420.8Fall 2007

Nan Yu

Survey Research

3

Picking a Research Design

Quantitative methodssurveyexperimentcontent analysis

Each one of the above-mentioned designs have advantages and disadvantages, and based on what type of question a researcher is trying to answer, one is more appropriate than the others.

4

Survey

“Audience research”

Questioning people about their attitudes, emotions, beliefs, intentions, and behaviors.

How people evaluate or perceive issue, event or message.

5

Characteristics of Survey

A set of questions to measure variables

Researchers tend to find the patterns among variables.

Answers “why people differ in their perceptions of an event.”

6

Difference between poll and survey

Poll Survey

short long, in-depth

what participants thinkwhy people have different thoughts

determine the frequency distribution of the values

of one variable

determine the relationship between two or more

variables.

7

Poll vs. Survey

Poll Survey

38 4

2 40

Dem

Attitude toward abortion

Rep

Pro-Choice

Pro-Life

•95.2% of Republicans are pro-life.•90.5% of Democrats are pro-choice.The data suggests that there is a relationship between political affiliation and attitude toward abortion.

Pro-life: 51%Pro-choice: 49%

8

Sampling (a review)

Census: if all people in the population were surveyed

SamplingNon-probabilityProbability

Sampling errorSampling process (sometimes random)biases

9

Sampling error (standard error)

Sampling error gives us some ideas of the estimated amount of errors you are willing to tolerate in making your selection.

Calculate sampling error is based on probability theory.Simple random, systematic, and stratified

samples

Sometimes researchers report sampling error if the sampling is close to probability sample.

10

How to calculate sampling error

NPPSE /)1( SE = Sampling error

P= Sample proportion (population parameters of interests)

The proportion of people in the population being surveyed who are expected to answer a certain way on the key measure in the survey. If you are unsure what the proportion might be, use 50% because this produces the maximum possible variation.

N=Sample size

11

Example of sampling-error calculation

A poll conducted before the last presidential elections showed that 48% voters wanted to vote for Bush, 52% voters wanted to vote for Gore. The poll sampled 100 people from State College.

%5%99.4100/)%481(*%48 SE

5% sampling error suggests: In reality, the % of votes in the population can be anywhere between (43%, 53%) for Gore and between (47%, 57%) for Bush.

%5 interval confidence

12

Example (cont.)

Imagine that we now take a sample of 1000 respondents, and we find the same percentage of votes for Bush (48%) and Gore (52%).

1.4% sampling error suggests:

Range between which the true population values would fall would then be (46.6%; 49.4%) votes for Gore, and (50.6%; 53.4%) % votes for Bush.

%4.10100/)%481(*%48 SE

%4.1 interval confidence

13

Sampling Error (cont.)

The larger the sample size, the smaller the sampling error.

How would you know then what sample size would be sufficient? (p. 224)Depends on how accurate you want to be.

14

Sampling error and sample size

Sample size

The greater the sample size, the less the sampling error.

15

Sampling error

CensusIf you take a sample that consists of the entire

population you actually have no sampling error because you don't have a sample, you have the entire population.

16

Confidence level

The confidence level tells you how sure you can be. The degree to which that you are confident about the sample can represent the true population.

Most researchers use the 95% confidence level.

17

Confidence level (p.224)

If the population proportion is determined (50-50%, 30-70%, etc…)

Sample size

Sampling error (confidence interval)

Confidence level

18

Survey design

Based on the point in time when the questionnaire is administered, survey studies can be divided into: Cross-sectional studies

observation at one point of time

Longitudinal studies observation over time

establish patterns and shifts over time

19

Types Longitudinal Study

Trend Cohort Panel

different samplesSamples are

subgroups from a same population.

same samples

A cohort is a group of people who share a common characteristic or experience within a defined time period. (e.g. class of 1997, people that were born between 1990 to 1995. )

Sample size might be different over time (people moved, died…).

Require probability sampling from the same population, otherwise you can not compare the data collected at different times.

20

Types of surveys

Watt, J. H., & van den Berg, S. A. (2002). Research methods for communication science. Needham Heights, MA: Allyn and Bacon, p.278, p. 290

21

Constructing questionnaires

Make a list of the variables that you want to measure.

Construct questionsTypes of questions

Open-ended, closed questionsLikert-type, semantic differential

Levels of measurementNominal, ordinal, interval, or ratio

22

Constructing questionnaires

Principles: questions need to be direct, clear, and unambiguous.

Avoid double-barreled questions.

• Does your company offer pension and health insurance benefits?

• Are you a republican or a democrat?

Avoid double negations. They are grammatically incorrect and usually very confusing.

23

Principles of constructing questionnaires

Avoid leading questions (i.e., implying the answers in the questions)Examples:

• Isn’t true that PSU tuition is too high? (implies that it is too high)

• What are the flaws in the nation’s health-care system? (implies that the system has flaws)

24

Principles of constructing questionnaires

PRETEST before using it on a large scale (this will show you if there are any ambiguities, etc.)

Keep it short, so that you avoid respondents’ fatigue (assure that the last questions are answered as well as the first ones)

Avoid jargon, slang, and abbreviations

25

Problematic questions

What are problems with the following questions? “What is your income?”

“Do you jog regularly?”

“ Does your college offer master or doctoral programs?”

What is your total family’s household income before taxes during last year?

How many times do you job per week?1 2 3 4 5 6 7

Double-barreled questions

26

Sensitive questions

Researchers ask sometimes about sensitive/threatening issuessexual behaviordrug or alcohol abusedeviant behaviormental health illegal activitycontroversial public issuesdemographic questions (age, gender, ethnicity, income)

Researchers who ask such questions have to do so with extra care.

27

Techniques of asking sensitive questions

Goal: to increase truthful answers to sensitive questions:

“Foot in the door”

The researcher builds some trust and respondents are more comfortable towards the end with the researcher and the procedures.

Place sensitive questions (e.g. demographic questions) at the end.

28

Techniques of asking sensitive questions

Assuring the anonymity of the respondent.

Be careful with wording (e.g., forbidden versus not allowed)

Self-administered questionnaires (mail or internet surveys.

29

Closed Questions:

Easy to response (answers are provided)

If participants don’t have an opinion…

Good for sensitive questions

If the provided answers are not exhaustive…

Fewer irrelevant or confused answers

You never know if one is misunderstand the

question…

Easy to replicateSimplistic answers to

complex issues

Advantages Disadvantages

30

Open QuestionsAdvantages Disadvantages

Unlimited answers Hard to code/comparison

More detailed answers to complex issues

Respondents may be intimidated by the question

Unanticipated answers

(important to new measurements)

Different respondents give different degrees of detail in

answers

Reveal participants’ thinking process

Require more time and thoughts

31

SurveysAdvantages Disadvantages

Surveys show correlational evidence

(i.e., patterns of variables in a large population).

Survey instruments rely solely on self-reported

measures. Low reliability.

Surveys are very good at discerning attitudes, beliefs

and perceptions

They are not good for studying behavior,

emotions, etc.

Have high external validity

(real-world situation)

They can hardly prove causation.

Random error is high.

32

In-class demo

Researchers want to design a study to see PSU students’ thoughts on campus bus service.

Researchers assume that that 50% respondents think the campus bus service needs improvement, 50% students are satisfied with the service.

Researchers decide that they can tolerate if the sampling error is 3%. The confidence level in this study is 95%.

According to the table on p. 224, how many people should the researchers sample?

33

Answer to in-class demo

According to the table on p. 224, researchers have to sample 1,067 people.