1 Multifactor ANOVA. 2 What We Will Learn Two-factor ANOVA K ij =1 Two-factor ANOVA K ij =1...
-
date post
19-Dec-2015 -
Category
Documents
-
view
219 -
download
5
Transcript of 1 Multifactor ANOVA. 2 What We Will Learn Two-factor ANOVA K ij =1 Two-factor ANOVA K ij =1...
11
Multifactor ANOVAMultifactor ANOVA
22
What We Will LearnWhat We Will Learn• Two-factor ANOVA KTwo-factor ANOVA Kijij=1=1
– InteractionInteraction– Tukey’s with multiple Tukey’s with multiple
comparisonscomparisons– Concept of randomized Concept of randomized
blocked experimentsblocked experiments– Random effects and mixed Random effects and mixed
modelsmodels
• Two-factor ANOVA KTwo-factor ANOVA Kijij>1>1– InteractionsInteractions– Tukey’sTukey’s– Mixed and random effectsMixed and random effects
• Three-factor Three-factor ANOVAANOVA– Latin SquaresLatin Squares
• 22pp Factorial Factorial ExperimentsExperiments– 2233 Experiments Experiments– 22pp>3>3– Concept of Concept of
confoundingconfounding
33
2 Factor ANOVA2 Factor ANOVA
• Factor A consists of I levelsFactor A consists of I levels
• Factor B consists of J levelsFactor B consists of J levels
• IJ different pairsIJ different pairs
• Number of observations per each Number of observations per each factor pair Kfactor pair Kijij=1=1
• Example - TiresExample - Tires
Factor B - LocationFactor A - Brand Front Right Front Left Rear Right Rear Left
GoodyearFirestone
KellyMichelon
44
TerminologyTerminology• XXijij = the rv denoting the measurement when = the rv denoting the measurement when
factor A is held at level factor A is held at level ii and factor B is held at and factor B is held at level level jj
• xxijij = actual observed value = actual observed value
• The average when factor The average when factor AA is isheld at level held at level ii
• The average when factor The average when factor BB is is held at level held at level jj
• The grand meanThe grand mean
IJ
X
X
I
XX
J
X
X
J
jij
I
i
I
iij
j
J
jij
i
11..
1.
1.
55
An Additive ModelAn Additive ModelI I parametersparameters 11, , 22, , 33……II
J J parametersparameters 11, , 22, , 33, …, …jj
ST ST ijij = = ii + + jj
ijij is the sum of an effect due to factor A at level i ( is the sum of an effect due to factor A at level i (II) and an effect due to ) and an effect due to factor B at level j (factor B at level j (jj) )
Then XThen Xijij = = ii + + jj + + ijij
Number of model parameters Number of model parameters I+J+1I+J+1
One for the errorOne for the error
ijij- - i'ji'j = ( = (ii + + jj ) - ( ) - (i'i' + + jj )= )= ii - - i’i’
The difference in mean responses for two levels of one of the factors is the The difference in mean responses for two levels of one of the factors is the same for all levels of the other factor same for all levels of the other factor
ii and and jj are not uniquely defined are not uniquely defined
66
Additive ModelAdditive Model
XXijij = = + + ii + + jj + + ij ij wherewhereand and ijij’s are independent and N(0,’s are independent and N(0,22))
Now Now = 4, = 4, 11 = -.5, = -.5, 22 = .5, = .5, 11 = -1.5, = -1.5, 22 = 1.5 = 1.5
The parameters are uniquely defined The parameters are uniquely defined Have (I-1)+(J-1)+1 = I+J-1 Have (I-1)+(J-1)+1 = I+J-1
EstimatorsEstimators
J
jji
I
ii and
11
00
1 = 2
2 = 4
1 = 1 = 2 = 5
2 = 2 = 3 = 6
..ˆ..ˆ..ˆ .. XXXXX jji
77
Two Factor HypothesisTwo Factor Hypothesis
0:
0:
0:
0:
321
321
oneleastatH
H
oneleastatH
H
aB
JioB
aA
IioA
88
Two-way ANOVATwo-way ANOVA
)1)(1(
111
111
11
1 1
2
....
2..
1 1
2.
1
2
...
2..
1 1
2.
1
2
...
1 1
2..
2
1 1
2
..
JIdf
XXXXSSBSSASSTSSE
JdfXIJ
XI
XXSSB
IdfXIJ
XJ
XXSSA
IJdfXIJ
XXXSST
I
i
J
j
jiij
I
i
J
jj
J
j
j
I
i
I
ii
J
j
i
I
i
J
jij
I
i
J
jij
99
ANOVA TableANOVA TableSource Sum of
SquaresDegrees ofFreedom
MeanSquare
F F,v1,v2 *?
Factor A SSA df=I-1 MSA FA F,dfA,dfE
Factor B SSB df=J-1 MSB FB F,dfB,dfE
Error SSE df=(I-1)(J-1) MSE
Total SST df=IJ-1
1010
Problem 1Problem 1A\B 1 2 3 4
1 200 226 240 261
2 278 312 330 381
3 369 416 462 517
4 500 575 645 733
1111
Problem 1Problem 1
)1)(1(
111
111
11
1 1
2
....
2..
1 1
2.
1
2
...
2..
1 1
2.
1
2
...
1 1
2..
2
1 1
2
..
JIdf
XXXXSSBSSASSTSSE
JdfXIJ
XI
XXSSB
IdfXIJ
XJ
XXSSA
IJdfXIJ
XXXSST
I
i
J
j
jiij
I
i
J
jj
J
j
j
I
i
I
ii
J
j
i
I
i
J
jij
I
i
J
jij
1212
Problem 1Problem 1Source Sum of
SquaresDegrees ofFreedom
MeanSquare
F F,v1,v2 *?
Factor A SSA df=I-1 MSA FA F,dfA,dfE
Factor B SSB df=J-1 MSB FB F,dfB,dfE
Error SSE df=(I-1)(J-1) MSE
Total SST df=IJ-1
1313
SPSS Data EntrySPSS Data Entry Analyze> General Linear Model (GLM)>UnivariateAnalyze> General Linear Model (GLM)>Univariate
1414
Problem 1Problem 1
Tests of Between-Subjects Effects
Dependent Variable: HEAT_TF
2960142.938a 7 422877.563 412.248 .000
39934.188 3 13311.396 12.977 .001
324082.188 3 108027.396 105.312 .000
9232.063 9 1025.785
2969375.000 16
SourceModel
LIQUID
GAS
Error
Total
Type III Sumof Squares df Mean Square F Sig.
R Squared = .997 (Adjusted R Squared = .994)a.
1515
Your TurnYour Turn
• The effects of four types of graphite coaters The effects of four types of graphite coaters on light box readings are to be studied. As on light box readings are to be studied. As these readings might differ from day to these readings might differ from day to day, observations are to be taken on each day, observations are to be taken on each of the four types everyday for three days. of the four types everyday for three days. The order of testing of the four types on The order of testing of the four types on any given day can be randomized.any given day can be randomized.
• Analyze these data as a randomized block Analyze these data as a randomized block design and state your conclusions.design and state your conclusions.
1616
Your TurnYour Turn
Day M A K L
1 4.0 4.8 5.0 4.6
2 4.8 5.0 5.2 4.6
3 4.0 4.8 5.6 5.0
1717
Your TurnYour TurnSource Sum of
SquaresDegrees ofFreedom
MeanSquare
F F,v1,v2 *?
Factor A
Factor B
Error
Total
1818
Multiple Comparisons in Multiple Comparisons in ANOVAANOVATukey’s Procedure (T Tukey’s Procedure (T Method)Method)
• SelectSelect • Determine QDetermine Q
– For A - QFor A - Q,I,(I-1)(J-1),I,(I-1)(J-1)
– For B - QFor B - Q,J,(I-1)(J-1),J,(I-1)(J-1)
• Determine Determine ww for factors A and B for factors A and B
• List sample means in increasing order List sample means in increasing order
• Compute the difference in each Compute the difference in each ii - -j j pair Underline pair Underline those pairs that differ by less than those pairs that differ by less than wwPairs Pairs notnot underlined are significantly different underlined are significantly different
I
MSEQw
J
MSEQw
JIJB
JIIA
)1)(1(,,
)1)(1(,,
1919
Problem 1Problem 1
• SelectSelect • Determine QDetermine Q
– For A - QFor A - Q,I,(I-1)(J-1),I,(I-1)(J-1)
– For B - QFor B - Q,J,(I-1)(J-1),J,(I-1)(J-1)
• Determine Determine ww for factors A and B for factors A and B
• List sample means in increasing order List sample means in increasing order
• Compute the difference in each Compute the difference in each ii - -j j pair pair
• Underline those pairs that differ by less than Underline those pairs that differ by less than ww
I
MSEQw
J
MSEQw
JIJB
JIIA
)1)(1(,,
)1)(1(,,
2020
Problem 1Problem 1Multiple Comparisons
Dependent Variable: HEAT_TF
Tukey HSD
-45.5000 22.6471 .254 -140.8949 49.8949
-82.5000 22.6471 .023 -177.8949 12.8949
-136.2500* 22.6471 .001 -231.6449 -40.8551
45.5000 22.6471 .254 -49.8949 140.8949
-37.0000 22.6471 .408 -132.3949 58.3949
-90.7500 22.6471 .013 -186.1449 4.6449
82.5000 22.6471 .023 -12.8949 177.8949
37.0000 22.6471 .408 -58.3949 132.3949
-53.7500 22.6471 .152 -149.1449 41.6449
136.2500* 22.6471 .001 40.8551 231.6449
90.7500 22.6471 .013 -4.6449 186.1449
53.7500 22.6471 .152 -41.6449 149.1449
(J) LIQUID2.00
3.00
4.00
1.00
3.00
4.00
1.00
2.00
4.00
1.00
2.00
3.00
(I) LIQUID1.00
2.00
3.00
4.00
MeanDifference
(I-J) Std. Error Sig. Lower Bound Upper Bound
99% Confidence Interval
Based on observed means.
The mean difference is significant at the .01 level.*.
2121
Problem 1Problem 1
HEAT_TF
Tukey HSDa,b
4 336.7500
4 382.2500 382.2500
4 419.2500 419.2500
4 473.0000
.023 .013
LIQUID1.00
2.00
3.00
4.00
Sig.
N 1 2
Subset
Means for groups in homogeneous subsets are displayed.Based on Type III Sum of SquaresThe error term is Mean Square(Error) = 1025.785.
Uses Harmonic Mean Sample Size = 4.000.a.
Alpha = .01.b.
2222
Problem 1Problem 1Multiple Comparisons
Dependent Variable: HEAT_TF
Tukey HSD
-93.5000 22.6471 .011 -188.8949 1.8949
-209.2500* 22.6471 .000 -304.6449 -113.8551
-381.5000* 22.6471 .000 -476.8949 -286.1051
93.5000 22.6471 .011 -1.8949 188.8949
-115.7500* 22.6471 .003 -211.1449 -20.3551
-288.0000* 22.6471 .000 -383.3949 -192.6051
209.2500* 22.6471 .000 113.8551 304.6449
115.7500* 22.6471 .003 20.3551 211.1449
-172.2500* 22.6471 .000 -267.6449 -76.8551
381.5000* 22.6471 .000 286.1051 476.8949
288.0000* 22.6471 .000 192.6051 383.3949
172.2500* 22.6471 .000 76.8551 267.6449
(J) GAS2.00
3.00
4.00
1.00
3.00
4.00
1.00
2.00
4.00
1.00
2.00
3.00
(I) GAS1.00
2.00
3.00
4.00
MeanDifference
(I-J) Std. Error Sig. Lower Bound Upper Bound
99% Confidence Interval
Based on observed means.
The mean difference is significant at the .01 level.*.
2323
Problem 1Problem 1HEAT_TF
Tukey HSDa,b
4 231.7500
4 325.2500
4 441.0000
4 613.2500
.011 1.000 1.000
GAS1.00
2.00
3.00
4.00
Sig.
N 1 2 3
Subset
Means for groups in homogeneous subsets are displayed.Based on Type III Sum of SquaresThe error term is Mean Square(Error) = 1025.785.
Uses Harmonic Mean Sample Size = 4.000.a.
Alpha = .01.b.
2424
Your TurnYour Turn
• SelectSelect • Determine QDetermine Q
– For A - QFor A - Q,I,(I-1)(J-1),I,(I-1)(J-1)
– For B - QFor B - Q,J,(I-1)(J-1),J,(I-1)(J-1)
• Determine Determine ww for factors A and B for factors A and B
• List sample means in increasing order List sample means in increasing order
• Compute the difference in each Compute the difference in each ii - -j j pair pair
• Underline those pairs that differ by less than Underline those pairs that differ by less than ww
I
MSEQw
J
MSEQw
JIJB
JIIA
)1)(1(,,
)1)(1(,,
2525
Concept of Randomized Block Concept of Randomized Block DesignsDesigns• 1-way ANOVA1-way ANOVA
– II treatments treatments– IJIJ total observations or subjects total observations or subjects– JJ observations per treatment selected randomly observations per treatment selected randomly
• Observations or subjects can be heterogeneous WRT other Observations or subjects can be heterogeneous WRT other variablesvariables– Significance or non-significance due to the treatment or something Significance or non-significance due to the treatment or something
elseelse– Hence paired t-testHence paired t-test
•pre to post exams - looking at the difference of an individual’s scores; know pre to post exams - looking at the difference of an individual’s scores; know that the difference isn’t due to the subjectthat the difference isn’t due to the subject
• When When II>2 we want to perform a >2 we want to perform a randomized blockrandomized block experiment experiment
2626
Concept of Randomized Concept of Randomized Block DesignsBlock Designs
• The extra factor (block) divides the The extra factor (block) divides the IJIJ units into units into JJ groups with groups with II units within units within each groupeach group– The The II units are homogeneous WRT other factors (the block) units are homogeneous WRT other factors (the block)– Within each block, the Within each block, the II treatments are randomly selected and assigned to treatments are randomly selected and assigned to II
observations or subjectsobservations or subjects
– When social scientists use this - repeated measuresWhen social scientists use this - repeated measures– Each subject undergoes each treatment; thus, acting as their own controlEach subject undergoes each treatment; thus, acting as their own control– Could use time periods, locations, etc.Could use time periods, locations, etc.– From a large population of subjects - random effectsFrom a large population of subjects - random effects
Source Sum ofSquares
Degrees ofFreedom
MeanSquare
F F,v1,v2 *?
Factor A SSA df=I-1 MSA FA F,dfA,dfE
Block SSBlk df=J-1 MSBlk FBlk F,dfBlk,dfE
Error SSE df=(I-1)(J-1) MSE
Total SST df=IJ-1
2727
Problem 2Problem 2Subject
Seat 1 2 3 4 5 6 7 8 9
1 12 10 7 7 8 9 8 7 9
2 15 14 14 11 11 11 12 11 13
3 12 13 13 10 8 11 12 8 10
4 10 12 9 9 7 10 11 7 84
2828
Why Block?Why Block?• If the experimental units are heterogeneousIf the experimental units are heterogeneous
– Then there will be a larger variance in MSEThen there will be a larger variance in MSE– Blocking minimizes the random (MSE) variance by Blocking minimizes the random (MSE) variance by
accounting some of the error to the effects due to the accounting some of the error to the effects due to the subjects/experimental unitssubjects/experimental units
– Thus, MSE (the random error) will be smaller allowing us to Thus, MSE (the random error) will be smaller allowing us to determine if there is significance in the main effect or determine if there is significance in the main effect or treatment.treatment.
– So…. We may be able to determine if the null hypothesis So…. We may be able to determine if the null hypothesis should be rejected.should be rejected.
– There is a cost…. The MSE will have fewer degrees of There is a cost…. The MSE will have fewer degrees of freedom since some of those df’s need to go to the Block freedom since some of those df’s need to go to the Block error (the larger the df, the smaller the MSE) error (the larger the df, the smaller the MSE)
2929
Models for Random EffectsModels for Random Effects• 1-way ANOVA - Random Effects Model1-way ANOVA - Random Effects Model
• 2-way ANOVA - Mixed Effects Model2-way ANOVA - Mixed Effects Model– XXijij = = + + ii + + BBjj + + ijij
– Distributions of Distributions of BBjj & & ijij are N(0, are N(0, 22,B ,B ), ),
N(0, N(0, 22,,,,) respectively) respectively
– Hypothesis TestHypothesis Test
I
ii
1
0
0: 2 BoBH 0:
0: 321
oneleastatH
H
aA
IioA
3030
2-Factor ANOVA 2-Factor ANOVA KKijij>1>1• Here, the responses are notHere, the responses are not
additiveadditive
• There is something going onThere is something going onbetween factor A and factor Bbetween factor A and factor B
• When additivity doesn’t applyWhen additivity doesn’t applywe have an interactionwe have an interaction
• Additivity allows us to obtain an unbiased estimator for MSEAdditivity allows us to obtain an unbiased estimator for MSE
• Need to have more than one observation per cell to find a Need to have more than one observation per cell to find a unbiased estimator for MSE when interactions may be present unbiased estimator for MSE when interactions may be present
• KKijij>1 >1 – K- is a constant number per cell (each cell has same number of K- is a constant number per cell (each cell has same number of
observations)observations)
3131
Parameters for Fixed Effects Parameters for Fixed Effects Model w/InteractionModel w/Interaction
IJ
I
J
J
jij
I
i
I
iij
j
J
jij
i
11..
1.
1.
ii = = i.i.-- = = the effect of the effect of factor A at level factor A at level II
jj = = ..j j -- = = the effect of the effect of factor B at level factor B at level JJ
ij ij is the interaction is the interaction parameters parameters
ijij = = ijij – ( – ( + + ii + + jj))
so the individual means are so the individual means are represented belowrepresented below
ijij = = + + ii + + jj + + ijij
3232
HypothesesHypotheses
0:
,0:
0:
0:
0:
0:
321
321
ijaA
ijoAB
aB
IoB
aA
IoA
oneleastatH
jiallforH
oneleastatH
H
oneleastatH
H
• The model is The model is additive if all additive if all ijij=0=0
• Order of testingOrder of testing– Interaction firstInteraction first– Main effectsMain effects
• Sometimes results Sometimes results can be confusingcan be confusing
3333
The Model with Interactions The Model with Interactions • XXijkijk = the rv denoting the = the rv denoting the
measurement when measurement when factor A is held at level factor A is held at level ii and factor B is held at and factor B is held at level level j j given given kk observations for each observations for each ij ij levelslevels
• xxijkijk = actual observed = actual observed valuevalue
• XXijkijk = = + + ii + + jj + + ij ij + + ijij
ij ij are N(0,are N(0, 22))
3434
The Model with Interactions The Model with Interactions
)1)(1(
1
)1)(1(
111
111
11
1 1 1 1
2.
1
22.
1
1 1
2
........1
2...
1 1
2..
1
2
.....
1
2...
1 1
2..
1
2
.....
1
1 1
2...
2
11 1
2
...1
JIdf
XK
XXX
SSABSSBSSASSTSSE
JIdfXXXXSSAB
JdfXIJK
XIK
XXSSB
IdfXIJK
XJK
XXSSA
IJKdfXIJK
XXXSST
I
i
J
j
I
i
J
j i jij
K
kijkijijk
K
k
I
i
J
j
jiij
K
k
I
i
J
jj
J
j
j
K
k
I
i
I
ii
J
j
i
K
k
I
i
J
jijk
K
k
I
i
J
jijk
K
k
3535
The ANOVA TableThe ANOVA TableSource Sum ofSquares
Degrees ofFreedom
MeanSquare
F F,v1,v2 *?
Factor A SSA df=I-1 MSA FA F,dfA,dfE
Factor B SSB df=J-1 MSB FB F,dfB,dfE
A*B SSAB df=(I-1)(J-1) MSAB FAB F,dfAB,dfE
Error SSE df=IJ (K-1) MSE
Total SST df=IJK-1
3636
Problem 2Problem 2Tests of Between-Subjects Effects
Dependent Variable: CURRENT
14627.778a 5 2925.556 42.128 .000
1237688.889 1 1237688.889 17822.720 .000
1244.444 2 622.222 8.960 .004
13338.889 1 13338.889 192.080 .000
44.444 2 22.222 .320 .732
833.333 12 69.444
1253150.000 18
15461.111 17
SourceCorrected Model
Intercept
PHOSPHOR
GLASS
PHOSPHOR * GLASS
Error
Total
Corrected Total
Type III Sumof Squares df Mean Square F Sig.
R Squared = .946 (Adjusted R Squared = .924)a.
3737
Problem 2Problem 2
Estimated Marginal Means of CURRENT
GLASS
2.001.00
Est
ima
ted
Ma
rgin
al M
ea
ns
320
300
280
260
240
220
PHOSPHOR
1.00
2.00
3.00
3838
Multiple Comparisons in Multiple Comparisons in ANOVAANOVATukey’s Procedure (T Method)Tukey’s Procedure (T Method)• HHoABoAB is not rejected and H is not rejected and HoAoA and/or H and/or HoBoB is rejected is rejected
• SelectSelect • Determine QDetermine Q
– For A - QFor A - Q,I,IJ(K-1),I,IJ(K-1)
– For B - QFor B - Q,J,IJ(K-1),J,IJ(K-1)
• Determine Determine ww for factors A and B for factors A and B
• List sample means in increasing order List sample means in increasing order
• Compute the difference in each Compute the difference in each ii - -j j pair Underline those pairs pair Underline those pairs that differ by less than that differ by less than wwPairs Pairs notnot underlined are significantly different underlined are significantly different
IK
MSEQw
JK
MSEQw
KIJJB
KIJIA
)1(,,
)1(,,
3939
Problem 2Problem 2Multiple Comparisons
Dependent Variable: CURRENT
Tukey HSD
-13.3333 4.8113 .042 -30.5003 3.8336
6.6667 4.8113 .379 -10.5003 23.8336
13.3333 4.8113 .042 -3.8336 30.5003
20.0000* 4.8113 .004 2.8331 37.1669
-6.6667 4.8113 .379 -23.8336 10.5003
-20.0000* 4.8113 .004 -37.1669 -2.8331
(J) PHOSPHOR2.00
3.00
1.00
3.00
1.00
2.00
(I) PHOSPHOR1.00
2.00
3.00
MeanDifference
(I-J) Std. Error Sig. Lower Bound Upper Bound
99% Confidence Interval
Based on observed means.
The mean difference is significant at the .01 level.*.
4040
Problem 2Problem 2CURRENT
Tukey HSDa,b
6 253.3333
6 260.0000 260.0000
6 273.3333
.379 .042
PHOSPHOR3.00
1.00
2.00
Sig.
N 1 2
Subset
Means for groups in homogeneous subsets are displayed.Based on Type III Sum of SquaresThe error term is Mean Square(Error) = 69.444.
Uses Harmonic Mean Sample Size = 6.000.a.
Alpha = .01.b.
4141
Mixed and Random Effects Mixed and Random Effects ModelsModels• 1-way ANOVA - Random Effects Model1-way ANOVA - Random Effects Model
• 2-way ANOVA - Mixed Effects Model2-way ANOVA - Mixed Effects Model– XXijij = = + + ii + + BBjj + + GGijij + + ijij
– Distributions of Distributions of BBjj, G, Gijij & &ijij are N(0, are N(0, 22,B ,B ), N(0, ), N(0, 22
,G ,G ), N(0, ), N(0, 22,,,,) )
respectivelyrespectively– Hypothesis TestHypothesis Test
– Order of testingOrder of testing• Interaction firstInteraction first
•Main effectsMain effects
I
ii
1
0
0:
0:2
2
GoG
BoB
H
H
0:
0: 321
oneleastatH
H
aA
IioA
4242
Source Sum ofSquares
Degrees ofFreedom
MeanSquare
F F,v1,v2 *?
Factor AFixed
SSA df=I-1 MSA FAMSA/MSAB
F,dfA,dfE
Factor BRandom
SSB df=J-1 MSB FBMSB/MSE
F,dfB,dfE
A*BRandom
SSAB df=(I-1)(J-1) MSAB FABMSAB/MSE
F,dfAB,dfE
Error SSE df=IJ (K-1) MSE
Total SST df=IJK-1