Chapter 4How to get the Data Part1 In the first 3 lectures of this course we In the first 3 lectures of this course we

spoke at length about what care we spoke at length about what care we should take in conducting a study should take in conducting a study ourselves or in interpreting the results ourselves or in interpreting the results of someone else’s study.of someone else’s study.

What we didn’t mention was how to What we didn’t mention was how to actually conduct the study, that will be actually conduct the study, that will be the topic of today’s lecture.the topic of today’s lecture.

We saw that studies can take three We saw that studies can take three forms:forms:

ExperimentsExperiments Observational StudiesObservational Studies Sample SurveysSample Surveys

We saw earlier that experiments are We saw earlier that experiments are possibly the best way to conduct possibly the best way to conduct studies as the researcher usually has studies as the researcher usually has complete control over the elements complete control over the elements of the study. And experiments allow of the study. And experiments allow a determination of cause and effect.a determination of cause and effect.

Since this is a Psychology course Since this is a Psychology course and not a Biology or Chemistry and not a Biology or Chemistry course we will restrict ourselves to course we will restrict ourselves to experiments involving humans.experiments involving humans.

No not what you may think, No not what you may think, experiments these days rely on experiments these days rely on volunteer subjects. volunteer subjects.

Experiments

The Experimental procedure The Experimental procedure involves manipulating something involves manipulating something called the called the Explanatory VariableExplanatory Variable and seeing the effect on something and seeing the effect on something called the called the Outcome VariableOutcome Variable

Example:Example: In an experiment to test In an experiment to test the effect of a new drug designed to the effect of a new drug designed to reduce blood pressure, the reduce blood pressure, the explanatory variable would be the explanatory variable would be the amount of drug administered and the amount of drug administered and the outcome variable would be the outcome variable would be the reduction in blood pressure.reduction in blood pressure.

The experiment has to be designed The experiment has to be designed to eliminate to any extraneous effects to eliminate to any extraneous effects and to determine only the results of and to determine only the results of the explanatory variable on the the explanatory variable on the outcome variable.outcome variable.

The way that this is accomplished is The way that this is accomplished is that participants are randomly that participants are randomly assigned to one of two groups:assigned to one of two groups:

One group receives the treatment the One group receives the treatment the other receives a placebo - ie no other receives a placebo - ie no treatment at all. treatment at all.

This random assignment to a This random assignment to a treatment group or control group is treatment group or control group is the way most clinical trials are the way most clinical trials are conducted today. conducted today.

This form of study is similar to an This form of study is similar to an experiment except that the treatment experiment except that the treatment occurs naturally and is not imposed on occurs naturally and is not imposed on the subjects.the subjects.

It is much harder to establish a cause It is much harder to establish a cause and effect relationship using an and effect relationship using an observational study than using an observational study than using an experiment because we cannot create experiment because we cannot create control and treatment groups to control and treatment groups to eliminate confounding effects.eliminate confounding effects.

One attempt to isolate the explanatory One attempt to isolate the explanatory variable is to conduct what is called a variable is to conduct what is called a case control study.case control study.

We will examine this type of study in We will examine this type of study in detail later. detail later.

Observational Studies

FLUNK.NET'S STUDENT SEX FLUNK.NET'S STUDENT SEX SURVEYSURVEY

Total Poll: 1,041Total Poll: 1,041 Men: 603 (57.9%)Men: 603 (57.9%) Women: 438 (42.1%)Women: 438 (42.1%) Virgins: 142 (13.7%)Virgins: 142 (13.7%) Average age: 20Average age: 20

Average Age of Virginity Loss: Average Age of Virginity Loss: 1717

Average Age of Virginity Loss:Average Age of Virginity Loss: 17.317.3

Average Number of Sexual Average Number of Sexual Partners: 4.2Partners: 4.2

Average Number of Sexual Average Number of Sexual Partners 2.7Partners 2.7

% Who have had sex on Campus % Who have had sex on Campus 34%34%

% Who have had sex on Campus % Who have had sex on Campus approx 3%approx 3%

WEB vs 2000 Class

% Who would sleep with a % Who would sleep with a lecturer: 49%lecturer: 49%

% Who would sleep with a % Who would sleep with a lecturer: 49.5%lecturer: 49.5%

% Who have slept with someone % Who have slept with someone in their class: 43%in their class: 43%

% Who have slept with someone % Who have slept with someone in their class: approx 4%in their class: approx 4%

% Who would sleep with more % Who would sleep with more than one partner: 64%than one partner: 64%

% Who would sleep with more % Who would sleep with more than one partner: than one partner: ????????????

WEB vs 2000 Class

Some interesting facts

% Virgins:% Virgins: 31%31%

Non-Virgins: 14 0.01 15 0.01 16 0.22 17 0.32 18 0.27 19 0.13 20 0.02

Some interesting facts

# Partners 1 93 0.47 2 47 0.24 3 26 0.13 4 4 0.02 5 7 0.04 6 4 0.02 7 3 0.02 8 1 0.01 9 1 0.01 10 2 0.01 11 4 0.02 14 1 0.01 15 2 0.01 24 1 0.01 25 1 0.01

We will concentrate for the rest of We will concentrate for the rest of this lecture on sample surveys.this lecture on sample surveys.

First some definitions:First some definitions: AA Unit Unit is a single individual or is a single individual or

object to be measured.object to be measured. AA Population Population is the entire is the entire

collection of Units about which we collection of Units about which we would like information.would like information.

AA Sample Sample is the collection of Units is the collection of Units we actually measure.we actually measure.

AA Sampling Frame Sampling Frame is a list of Units is a list of Units from which the sample is chosen. from which the sample is chosen. Ideally the sampling frame Ideally the sampling frame includes the whole Population. includes the whole Population.

Sample Surveys

In aIn a Sample Survey Sample Survey measurements measurements are taken on a sample chosen from are taken on a sample chosen from the Populationthe Population

In a In a CensusCensus the entire Population is the entire Population is surveyed.surveyed.

Resources are needed to conduct a Resources are needed to conduct a CensusCensus

CSO Spends about £20million to CSO Spends about £20million to conduct the 5 year Census of conduct the 5 year Census of PopulationPopulation

Sometimes the measuring process Sometimes the measuring process destroys the thing being measured, destroys the thing being measured, e.g. if we were to test the e.g. if we were to test the strength strength of a weld or in testing an individuals of a weld or in testing an individuals blood - who among us would be willing blood - who among us would be willing to donate all of our blood in a test?to donate all of our blood in a test?

Because of the work involved in a Because of the work involved in a Census it is much faster to conduct a Census it is much faster to conduct a survey, sometimes it is important to survey, sometimes it is important to have results fast. have results fast.

Why is Sampling used

There are accuracy advantages to be There are accuracy advantages to be had in conducting a sample survey:had in conducting a sample survey: It is easier to get complete It is easier to get complete

coverage of a sample than of a coverage of a sample than of a population. population.

Easier to train a small number of Easier to train a small number of interviewers for a sample survey interviewers for a sample survey than to train a large number for a than to train a large number for a census. census.

OK but a sample is still a sample and OK but a sample is still a sample and is bound to be inaccurate by its very is bound to be inaccurate by its very nature, isn’t it???nature, isn’t it???

British General ElectionBritish General Election

Accuracy in Surveys

We mentioned before that if a sample We mentioned before that if a sample was chosen to be representative of was chosen to be representative of the target population then it could be the target population then it could be very accurate.How accurate?very accurate.How accurate?

For surveys conducted to measure a For surveys conducted to measure a sample proportion as an estimate for sample proportion as an estimate for a population proportion we can a population proportion we can define a define a Margin of ErrorMargin of Error..

The sample proportion will differ The sample proportion will differ from the population proportion by from the population proportion by more than the margin of error less more than the margin of error less than 5% of the time.than 5% of the time.

The Margin of Error for a sample of The Margin of Error for a sample of size n is size n is 1/1/nn

Accuracy in Sampling

For Example with a sample of 1600 For Example with a sample of 1600 the margin of error is 1/40 or 2.5%the margin of error is 1/40 or 2.5%

So a survey conducted using a So a survey conducted using a sample of size 1600 will be accurate sample of size 1600 will be accurate to within 2.5% more than 95% of the to within 2.5% more than 95% of the time.time.

Accuracy in Sampling

We saw already that in order for the We saw already that in order for the sampling procedure to work properly sampling procedure to work properly the sample must be representative of the sample must be representative of the target populationthe target population

There are several ways to get a There are several ways to get a representative sample:representative sample:

Simple Random SamplingSimple Random Sampling Stratified Random SamplingStratified Random Sampling Cluster SamplingCluster Sampling Systematic SamplingSystematic Sampling Random Digit DialingRandom Digit Dialing Multi Stage SamplingMulti Stage Sampling

How to choose a Sample

The simplest form of sampling The simplest form of sampling procedure.procedure.

Each group of individuals has the Each group of individuals has the same chance of getting chosen.same chance of getting chosen.

Therefore each individual has the Therefore each individual has the same chance of being chosen.same chance of being chosen.

Use Random Number Tables or a Use Random Number Tables or a random number generator.random number generator.

Or put names in a hat or roll a die.Or put names in a hat or roll a die.

Simple Random Sampling

Polling companies don’t have a list Polling companies don’t have a list of all adults and select from that list of all adults and select from that list randomly.randomly.

Instead they use other methods like Instead they use other methods like Stratified Random SamplingStratified Random Sampling

We first divide the population into We first divide the population into different strata, then sample different strata, then sample randomly within those strata.randomly within those strata.

Example: To conduct an opinion poll Example: To conduct an opinion poll we might divide the population into we might divide the population into different age groups or sexes or by different age groups or sexes or by County of residence.County of residence.

Stratified Random Sampling

Advantages of this method are:Advantages of this method are: We can get individual estimates for We can get individual estimates for

each strataeach strata We can use different interviewers for We can use different interviewers for

each strata and train them each strata and train them appropriatelyappropriately

If strata are different geographic If strata are different geographic regions it may be cheaper to sample regions it may be cheaper to sample them separately.them separately.

Stratified Random Sampling

Divide the Population into similar Divide the Population into similar groups or clusters. Then choose a groups or clusters. Then choose a random sample of clusters.random sample of clusters.

The analysis of this type of survey is The analysis of this type of survey is more complicated than for simple more complicated than for simple random sampling.random sampling.

NOTE: This is not the same as NOTE: This is not the same as Stratified Sampling, in Cluster Stratified Sampling, in Cluster Sampling the Clusters are chosen so Sampling the Clusters are chosen so that the resemble each other as much that the resemble each other as much as possible.as possible.

Cluster Sampling

This is where a plan is used to chose This is where a plan is used to chose the participants in the study.the participants in the study.

For Example: We might decide to For Example: We might decide to survey every 3rd person we meet. Or survey every 3rd person we meet. Or to choose every 5th House to be to choose every 5th House to be surveyed.surveyed.

Sometimes this procedure can be Sometimes this procedure can be very biased. very biased.

What happens if every 5th house is What happens if every 5th house is an end house?an end house?

Systematic Sampling

Used very much in the USUsed very much in the US Phone numbers in certain area codes Phone numbers in certain area codes

are dialled randomly by computer, are dialled randomly by computer, then when someone answers an then when someone answers an interviewer asks questionsinterviewer asks questions

Random Digit Dialling

Multi- Stage Sampling

Used for large surveysInvolves using a combination of the methods described above.

Here are 5 ways to make a mess of Here are 5 ways to make a mess of the sampling procedure:the sampling procedure:

Use the wrong sampling frameUse the wrong sampling frame Fail to reach the individuals selectedFail to reach the individuals selected Get no responseGet no response Get a sample of volunteersGet a sample of volunteers Use a convenient or haphazard Use a convenient or haphazard

samplesample

The last 2 of these are disastrousThe last 2 of these are disastrous

What can go wrong in Sampling

1936 US election1936 US election Literary Digest had been extremely Literary Digest had been extremely

successful in predictionssuccessful in predictions 1936 predicted 3-2 victory for Rep 1936 predicted 3-2 victory for Rep

Landon over Dem. FDRLandon over Dem. FDR George Gallup American Institute of George Gallup American Institute of

Public Opinion predicted FDR correctly Public Opinion predicted FDR correctly and also predicted what Literary Digest and also predicted what Literary Digest would predict.would predict.

Literary Digest 10 millionLiterary Digest 10 million Gallup 50,000Gallup 50,000 LD- Magazine Subscribers, Phone LD- Magazine Subscribers, Phone

Directories, Car Owners - WealthyDirectories, Car Owners - Wealthy Most serious though: 23% Volunteer Most serious though: 23% Volunteer

response.response.

What went wrong in Sampling