Producing Data: Samples and

Experiments

Chapter 4

Discussion example 1

One school board member noticed that students in band tended to be in the top 25% of their school. She compiled a list from each high school’s band director and took a random sample of 25 students from each school’s band. She then took a random sample of 25 students from each high school that wasn’t in band. She found a slightly higher average G.P.A. of student’s in band.

Discussion example 1

Will this study give evidence that being in

band causes an increase in a students

G.P.A?

Will this study help her generalize that

student’s in band tend to have a slightly

higher G.P.A. than students not in band?

Vocabulary from example 1

Observational study

a study based on data collected from individuals that

meet a determined criteria

Lurking variable (“extraneous variable”)

an outside factor that is not the explanatory (x-variable)

nor response (y-variable) variable

Variables we cannot control

In practice, we assume that all of these lurking

variables are randomly spread out equally enough to

not impact the response variable significantly

Discussion example 2

Another school board member is surprised

the increase is so slight. First, he emails

each band director and asks for a list of 30

students. He then accesses each high

school’s roster takes the first 40 listed

striking any student’s name has already has.

He found the average G.P.A. of student’s in

band to be more significant than the first

study.

Discussion example 2

Will this study give evidence that being in

band causes an increase in a students

G.P.A?

Will this study help her generalize that

student’s in band tend to have a slightly

higher G.P.A. than students not in band?

Discussion example 3

Walmart is considering buying a gasoline additive that is suppose to improve gas mileage. They found 30 employees in Texas that drive the same car. Fifteen employees are randomly selected to receive the additive, the remaining fifteen are given a bottle with just gas. Each employee is given a set route around the city to drive. The gas mileage is recorded by an onboard computer which shows the additive gives the driver 12% better gas mileage.

Discussion example 3

Will this study give evidence that using the additive will give a car better gas mileage?

Vocabulary from example 3

Experiment

a planned study where deliberate conditions are

imposed to see how the response variable will change

Confounding variable

a variable associated (noncausal) with the explanatory

variable that affects the response variable in some

way

makes it difficult to tell if the treatment or the

confounding variable affected the response variable

significantly

Lurking versus confounding

Observation study

x y

z

Lurking

Experiment

x y

z

Confounding

?

?

Randomized comparative

experiments

Goal of an experiment: collect statistically

significant evidence for a cause-and-effect

relationship.

Principles of Experimental Design

Control:

using comparison ensures that outside factors operate equally on all groups

comparison minimizes effects of confounding variables allowing effects of treatments to be better evaluated

control groups allow us to accurately assess the change in the response variable

Principles of Experimental Design

Control:

Randomization:

use of impersonal chance in order equalize unanticipated factors so that groups that should be similar in all respects.

Creates homogenous groups to reduce variability allowing better assessment of treatments

Principles of Experimental Design

Control:

Randomization:

Replication:

perform the experiment on as many subjects to reduce chance variation in the results

Experimental Design Examples

Read each design example and write a

description on how each experiment should

be run.

Key terms: groups, treatments, comparison,

randomization

Completely Randomized Design

Random

Allocation

Group 1

15 babies

Group 2

15 babies

Treatment 1

Her product

Treatment 2

Competitor’s

Compare

weight

gain

Babies will be numbered 01 to 30. Using a random number table, select

15 unique 2-digit numbers. The first 15 selected will be in Group 1

with the remaining placed in Group 2. Each babies’ weight will be

measured in pounds and compared.

Block Design

Subjects

African

American men

White men

Random

assignment

Random

assignment

Group 1

Group 2

Group 3

Group 4

Treatment 1

Calcium

Treatment 1

Calcium

Treatment 2

Placebo

Treatment 2

Placebo

Compare

blood

pressure

All African American men will be assigned a random unique number. Half the

men who have the smallest numbers will be assigned group 1, the half

with the largest numbers will be assigned group 2. The process will repeat

for the white men. The reduction in blood pressure will be compared.

Matched pair Design

Random

Allocation

Group 1

Group 2

Treatment 1

left hand

Treatment 2

right hand

Compare

difference

A coin will be flipped to decide which hand will be measured first by

each participant. Heads will squeeze the left hand first, tails will

squeeze the right hand first. The different in the pounds on the scale

will be compared.

Treatment 1

left hand

Treatment 2

right hand

Calcium experiment revisted

What potential problems might we have

because we started with random assignment?

How should we (did we) alter our experiment?

Random

Assignment

Group 1

Group 2

Treatment 1

Calcium

Treatment 2

Placebo

Compare

blood

pressure

Block Design

All participants

African

American men

White men

Completely randomized

experiment

Completely randomized

experiment

Big differences BETWEEN groups, but small

differences WITHIN groups

Improving the Design

A block is a group of experimental units or

subjects that are known before the experiment to

be similar in some way that is expected to affect

the response to the treatments.

Block design has the same rationale as a stratified

random sample.

Blocks allow us to reduce the amount of variation

to improve the accuracy of our conclusions by

creating homogeneous groups.

single blind versus double blind

Improving the Design

In a matched pair design, each subject in the experiment will receive two (and only two) treatments.

The order that each subject receives both treatments is randomly selected to preserve the important aspect of randomization.