18-1

Introduction to ExperimentalDesign and the Analysis of

Variance• Situations where comparing more than two means is

important.• Difference between a sampling study and an

experimental study.• An approach to testing equality of more than two

means.• Introduction to the simplest experimental design - the

Completely Randomized Design.

18-2

Study Designs and Analysis Approaches

• SRS from population with known σ- cts response.

• SRS from population withunknown σ - cts response.

• SRS with Bernouilli response.• SRSs from 2 popns with known σ.• SRSs from 2 popns with unknown

σ.• SRSs from 2 popns with Bernouilli

responses.• SRS from 1 popn with 2

responses.• SRS from 1 popn with 2 Bernouilli

responses.

• One sample z-test.

• One sample t-test.

• One sample Proportion test.

• Two sample z-test.

• Two sample t-test.

• Two sample proportion test.

• Matched samples t-test.

• Matched samples Proportion test.

18-3

Study Designs

Sampling Study - One sample drawn independently andrandomly from each of t > 2 populations. Our objective is tocompare the means of the t populations for statisticallysignificant differences in responses. Initially we will assumeall populations have common variance, later, we will test tosee if this is indeed true. (Homogeneity of variance tests).

Experimental Study - t>2 samples drawn independently andrandomly drawn from 1 population. Initially, each samplehas the same mean and variance (since drawn from thesame population). Separate treatments are applied to eachsample. A treatment is something done to the experimentalunits which would be expected to change the distribution(usually only the mean) of the response(s).

18-4

Sampling Study

Health Eaters

VegetariansMeat & Potato Eaters

RandomSample

CholesterolLevels

1n1

12

11

y

yy

M

RandomSample

RandomSample

2n2

22

21

y

yy

M

3n3

32

31

y

yy

M

18-5

Experimental Study

Male College Undergraduate Students

RandomSample #1

RandomSample #2

RandomSample #3

Health Diet Veg. Diet M & P Diet

CholesterolLevels @ 1year.

1n1

12

11

y

yy

M

2n2

22

21

y

yy

M

3n3

32

31

y

yy

M

18-6

Notation

Let yij be the value of the response for experimental unit j in group i.i=1,2, ..., tj=1,2, ..., ni iij )y(E µ=

differentaremeanstheofsome:H:H

a

0t210 µ=µ==µ=µ L

Let εij = yij - µi be the residual or deviation from the group mean.

Assuming yij ~ N(µi, σ2), then εij ~ N(0, σ2)

If H0 holds, yij = µ0 + εij , that is, all groups have the same mean andvariance.

18-7

Testing Approaches - Naive

Do all possible pair-wise t-tests.

320

310

210

:H:H:H

µ=µµ=µµ=µ

• Assume each test is performed at the α=0.05 level.• The probability of not rejecting Ho when Ho is true is 0.95 (1-α).• The probability of not rejecting Ho when Ho is true for all three tests is

(0.95)3 = 0.857.• Thus the true significance level for the overall test of no difference in

the means will be 1-0.857 = 0.143, NOT the α=0.05 level we thought itwould be.

In each individual t-test, only part of the information available toestimate the underlying variance is actually used. This is inefficient -WE CAN DO MUCH BETTER!

1

2

18-8

Testing Approaches - Analysis of VarianceThe term “analysis of variance” comes from the fact that this approachcompares the variability observed among sample means to a pooledestimate of the variability among observations within each group.

Within group variance is small compared to variability among means.Clear separation of means.

Within group variance is large compared to variability among means.Unclear separation of means.

x

y

-4 -3 -2 -1 0 1 2 3 4

x

y

-4 -3 -2 -1 0 1 2 3 4

x

y

-4 -3 -2 -1 0 1 2 3 4

x

y

-4 -3 -2 -1 0 1 2 3 4

x

y

-4 -3 -2 -1 0 1 2 3 4

x

y

-4 -3 -2 -1 0 1 2 3 4

18-9

Pooled Variance

From two-sample t-test with assumed equal variance, σ2, we produced apooled (within-group) sample variance estimate.

2nns)1n(s)1n(s

n1

n1s

yyt21

222

211

p

21p

21

−+−+−=

+

−=

Extend the concept of a pooled variance to t groups as follows:

tnSSW

)1n()1n()1n(s)1n(s)1n(s)1n(s

Tt21

2tt

222

2112

w −=

−++−+−−++−+−=

LL

If all the ni are equal to n then this reduces to an average variance.

∑=

=t

1i

2i

2w s

t1s

∑=

=t

1iiT nn

18-10

Variance among Group Means

If we assume each group is of the same size, say n, then under H0, s is anestimate of σ2/n. Hence, n times s is an estimate of σ2. When the samplesizes are unequal, the estimate is given by.

Consider the variance among the groupmeans computed as:

1

)(1

2

2

1

−

−=

=

∑

∑

=••

=••

t

yys

t

yy

t

ii

t

ii

11

)(1

2

2

−=

−

−=∑=

•••

tSSB

t

yyns

t

iii

B ∑ ∑

∑

= =••

=•

=

=t

1i

n

1jij

T

n

1jiji

i

i

yn1y

yy

18-11

F-test

Now we have two estimates of s2. An F-test can be used to determine ifthe two statistics are equal. Note that if the groups truly have differentmeans, sb

2 will be greater than sw2. Hence the F-statistics is written as:

)tn(),1t(2W

2B

TF~

ssF −−=

If H0 holds, the computed F-statistics should be close to 1.If Ha holds, the computed F-statistic should be much greater than 1.

We use the appropriate critical value from the F - table to help make this decision.

Hence,the F-test is really a test of equality of means under theassumption of normal populations and homogeneous variances.

18-12

Partition of Sums of Squaresand the AOV Table

SSBTSSSSW

ny

ny

yynSSB

ny

ysnyyTSS

t

i Ti

it

iii

T

t

i

n

jijT

t

i

n

jij

ii

−=

−=−=

−=−=−=

∑∑

∑ ∑∑ ∑

=

•••

=•••

••

= == =••

1

22

1

2

2

1 1

22

1 1

2

)(

)1()(

18-13

Spreadsheet ANOVATable 13.1 in Ott

Population1 2 3

5.90 5.51 5.015.92 5.50 5.005.91 5.50 4.995.89 5.49 4.985.88 5.50 5.02

mean 5.90 5.50 5.00variance 0.0003 0.0001 0.0003

standard deviation 0.0158 0.0071 0.0158n(1) 5.00 5.00 5.00

(n-1)var 0.00100 0.00020 0.00100

TSS 2.03553SSW 0.00220SSB 2.03333

ANOVA Table

SourceSums of Squares

Degrees of Freedom Mean Square F-test P-value

Between Samples 2.03333 2 1.01667 5545.45 0.000000Within Samples 0.00220 12 0.00018Total 2.03553 14

=average(b6:b10)=var(b6:b10)=sqrt(b13)=count(b6:b10)=(B15-1)*B13

=(sum(B15:D15)-1)*var(B6:D10)=sum(b16:d16)=b18-b19 =count(b5:d5)-1

=sum(b15:d15)-count(b5:d5)

18-14

The Linear Model

We have developed the one-way analysis of variance as an extension of thetwo-sample t-test with pooled variance. More complicated research designsrequire that we take a more formal, model-based approach to the analysis.

Much of statistical analysis is based on the general linear (regression) modelstructure. For the response yij for the ith group and jth individual or unit, wehave.

ijiijy ε+µ=Where µi is the mean of the ith group and eij is the deviations of the responsefrom the mean of the group.

Usual assumption: εij ~ N(0, s2) residual or experimental error

18-15

Completely Randomized Design

•Experimental Design - Completely randomized design (CRD)•Sampling Design - One-way classification design

Assumptions:• Independent random samples (results of one sample do not effect

other samples).• Samples from normal population(s).• Mean and variance for population i are respectively, µi and σ2.

Model: ijiijy ε+α+µ=

overall mean effect due to population irandom error ~ N(0,σ2)

AOV modeliij )y(E α+µ=

0fromdiffertheofoneleastAt:H0:H

a

t210

α=α==α=α L

Requirement for m to be the overall mean:

0t

1ii =α∑

=

18-16

Reference Group Model

Model:

1t,,2,1iytiy

ijitij

tjttj

−=ε+β+µ==ε+µ=

L

Mean for the last group (i=t) is µt.Mean for the first group (i=1) is µt + β1

Thus, β1 is the difference between themean of the reference group (cell) and thetarget group mean. Any group can be thereference group.

0fromdiffertheofoneleastAt:H0:H

a

1t210

β=β==β=β −L

reference group mean

effect due to population i

random error ~ N(0,σ2)This is the model SASuses.

Not in Ott.

18-17

Optional: Fixed versus Random Effects

Normally, the “effect” of a particular treatment, is assumed to be a constantvalue (αi) added to the response of all units in the group receiving the treatment.

If the treatments are well defined and easily replicable and is expected to produce thesame effect on average in each replicate, we have a fixed set of treatments and the AOVmodel is said to describe a fixed effects model.

Examples:• A scientist develops 3 new fungicides. Her interest is in these fungicides only.• The impact of 4 specific soil types on plant growth are of interest.• Three particular milling machines are being compared.• Four particular lakes are of interest in their weed biomass densities.

If the treatments cannot be assumed to be from a prespecified or known set oftreatments, they are assumed to be a random sample from some larger population ofpotential treatments. In this case, the AOV model is called a random effects model andthe αi are called random effects.

Examples:• A scientist is interested in how fungicides work. Ten (10) fungicides are selected (at

random) to represent the population of all fungicides in the research.• Four soil sub groups are selected for examining plant growth.• Three milling machines selected at random from the production line are compared.• 16 lakes selected at random are measured for their weed biomass densities.

18-18

Checking the Equal Variance Assumption2t

22

210 :H σ==σ=σ L Homogeneity of Variance

Hartley’s Test: A logical extension of the F test for t=2. Equal replication.

2min

2max

max ss

F = Reject if Fmax exceeds Fα,t,n-1 in Table 14

Bartlett’s Test: Unequal replication.

−−

−= ∑∑

==

t

1i

2iei

2e

t

1ii slog)1n(slog)1n(C ∑

==

t

1i

2i2

tss

If C > χ2(k-1),α then apply the

correction term

−−

−−+=

∑∑

=

=t

1ii

t

1i i )1n(

1)1n(

1)1t(3

11CF

Reject if C/CF > χ2(k-1),α

Levene’s Test:

18-19

Kruskal - Wallis Test

Extension of the rank-sum test for t=2 to the t>2 case.

H0: The t distributions are identical.Ha: Not all the distributions are the same.

Test Statistic: ∑=

+−+

=t

1i i

2i )1n(3

nT

)1n(n12H ∑

==

t

1iinn

Ti denotes the sum of the ranks for the measurements in sample iafter the combined sample measurements have been ranked.

Reject if H > χ2(t-1),α

With large numbers of ties in the ranks of the sample measurements use:

−−−

=∑

j

3j

3j )nn/()tt(1

H'H where tj is the number of observationsin the jth group of tied ranks.

18-20

options ls=78 ps=49 nodate;data OneWay; input popn resp @@ ; cards; 1 5.90 1 5.92 1 5.91 1 5.89 1 5.88 2 5.51 2 5.50 2 5.50 2 5.49 2 5.50 3 5.01 3 5.00 3 4.99 3 4.98 3 5.02;run;proc print; run;

Table 13.1 in Ott as a Reference Cell Model OBS POPN RESP 1 1 5.90 2 1 5.92 3 1 5.91 4 1 5.89 5 1 5.88 6 2 5.51 7 2 5.50 8 2 5.50 9 2 5.49 10 2 5.50 11 3 5.01 12 3 5.00 13 3 4.99 14 3 4.98 15 3 5.02

18-21

proc anova; class popn; model resp = popn ; title 'Table 13.1 in Ott - Analysis of Variance'; run;

Table 13.1 in Ott - Analysis of Variance 31 Analysis of Variance Procedure Dependent Variable: RESP Sum of Mean Source DF Squares Square F Value Pr > F Model 2 2.03333333 1.01666667 5545.45 0.0001 Error 12 0.00220000 0.00018333 Corrected Total 14 2.03553333 R-Square C.V. Root MSE RESP Mean 0.998919 0.247684 0.013540 5.466667 Source DF Anova SS Mean Square F Value Pr > F POPN 2 2.03333333 1.01666667 5545.45 0.0001

18-22

proc glm; class popn; model resp = popn /solution; title 'Table 13.1 in Ott’; run;

Table 13.1 in Ott 33 General Linear Models Procedure Dependent Variable: RESP Sum of Mean Source DF Squares Square F Value Pr > F Model 2 2.03333333 1.01666667 5545.45 0.0001 Error 12 0.00220000 0.00018333 Corrected Total 14 2.03553333 R-Square C.V. Root MSE RESP Mean 0.998919 0.247684 0.013540 5.466667 Source DF Type I SS Mean Square F Value Pr > F POPN 2 2.03333333 1.01666667 5545.45 0.0001 Source DF Type III SS Mean Square F Value Pr > F POPN 2 2.03333333 1.01666667 5545.45 0.0001 T for H0: Pr > |T| Std Error of Parameter Estimate Parameter=0 Estimate INTERCEPT 5.000000000 B 825.72 0.0001 0.00605530 POPN 1 0.900000000 B 105.10 0.0001 0.00856349 2 0.500000000 B 58.39 0.0001 0.00856349 3 0.000000000 B . . . NOTE: The X'X matrix has been found to be singular and a generalized inverse was used to solve the normal equations. Estimates followed by the letter 'B' are biased, and are not unique estimators of the parameters.