A Nova Presentation

7/24/2019 A Nova Presentation

1/163

ANalysis Of VAriance

Amit K Biswas

Indian Statistical Institute

at

Program for undergraduate students, Kozhikode

December 16, 2013

Amit (ISI, Chennai) ANOVA December 16, 2013 1 / 33
http://find/


2/163


Is a tool to test equality of mean values of several populations.

http://find/


3/163



However, this is achieved through a test of equality of two variances.

http://find/


4/163




So the original intended test is the following :

http://find/


5/163





H0 :1 =2 =3 = = k

H1 :i=j for i=j

http://find/


6/163





H0 :1 =2 =3 = = k

H1 :i=j for i=j

But the actual test carried out in ANOVA is :

http://find/


7/163





H0 :1 =2 =3 = = k

H1 :i=j for i=j


H0 :1 =2

H1 :i=j for i=j

http://goforward/http://find/http://goback/


8/163





H0 :1 =2 =3 = = k

H1 :i=j for i=j


H0 :1 =2

H1 :i=j for i=j

HOW is this ?

http://goforward/http://find/


9/163

TOH : a look back...

To test equality of two mean values we have several tests :

http://find/http://goback/


10/163



The ZTest

http://find/


11/163



The ZTest When variances are known,



12/163



The ZTest When variances are known, the students t-test,

http://find/


13/163



The ZTest When variances are known, the students t-test,whenvariances are unknown.

http://find/


14/163




H0 :1 =2H1 :1 =2



15/163




H0 :1 =2H1 :1 =2

H1 could also have > or< instead of not equal.

http://find/


16/163




H0 :1 =2H1 :1 =2


The test statistics being :X1 X22

1n1

+2

2n2

http://find/


17/163




H0 :1 =2H1 :1 =2



1n1

+2

2n2

which follows a normal distribution with mean zero and standard deviation1

http://find/


18/163




H0 :1 =2H1 :1 =2



1n1

+2

2n2

which follows a normal distribution with mean zero and standard deviation1

X1 X22

1n1

+2

2n2

N(0, 1)



19/163


and how do we test equality of two variances?

http://find/


20/163



H0 :21 =

22

H1 :21 =

22

http://find/


21/163



H0 :21 =

22

H1 :21 =

22

Here too, H1 could also have > or< instead of not equal.

http://find/


22/163



H0 :21 =

22

H1 :21 =

22


The test statistics...s21s22

which follows Fishers F distribution.

http://find/


23/163



H0 :21 =

22

H1 :21 =

22


The test statistics...s21s22

which follows Fishers F distribution.

Lets have a look at the TOH scenario in the tables next.

http://find/


24/163

Tests of Means

Tests for means with variance Unknown

Hypothesis Test Statistic Criteria for rejection

H0 : = 0 |t0| > t/2,n1H1 : = 0

H0 : = 0 t0 = y0

s/

n

H1 : < 0

http://find/


25/163

Tests of Means



H0 : = 0 |t0| > t/2,n1H1 : = 0

H0 : = 0 t0 = y0

s/

n

H1 : < 0 t0 < t,n1

H0 : = 0H1 : > 0

http://find/


26/163

Tests of Means



H0 : = 0 |t0| > t/2,n1H1 : = 0

H0 : = 0 t0 = y0

s/

n

H1 : < 0 t0 < t,n1

H0 : = 0H1 : > 0 t0 > t,n1H0 : 1 = 2H1 : 1 = 2

http://find/


27/163

Tests of Means



H0 : = 0 |t0| > t/2,n1H1 : = 0

H0 : = 0 t0 = y0

s/

n

H1 : < 0 t0 < t,n1

H0 : = 0H1 : > 0 t0 > t,n1H0 : 1 = 2H1 : 1 = 2 |t0| > t/2,

H0 : 1 = 2 t0 = y1y2

Sp

1n1

+ 1n2

H1 : 1 < 2 = n1+n2 2

http://find/


28/163

Tests of Means



H0 : = 0 |t0| > t/2,n1H1 : = 0

H0 : = 0 t0 = y0

s/

n

H1 : < 0 t0 < t,n1

H0 : = 0H1 : > 0 t0 > t,n1H0 : 1 = 2H1 : 1 = 2 |t0| > t/2,

H0 : 1 = 2 t0 = y1y2

Sp

1n1

+ 1n2

H1 : 1 < 2 = n1+n2 2 t0 < t,

if1 = 2H0 : 1 = 2

H1 : 1 > 2 =

S21n1

+S22n2

2

(S21/n1)2

n1+1 +

(S22/n2 )2

n2+1

2

http://find/


29/163

Tests of Means



H0 : = 0 |t0| > t/2,n1H1 : = 0

H0 : = 0 t0 = y0

s/

n

H1 : < 0 t0 < t,n1

H0 : = 0H1 : > 0 t0 > t,n1H0 : 1 = 2H1 : 1 = 2 |t0| > t/2,

H0 : 1 = 2 t0 = y1y2

Sp

1n1

+ 1n2

H1 : 1 < 2 = n1+n2 2 t0 < t,

if1 = 2H0 : 1 = 2

H1 : 1 > 2 =

S21n1

+S22n2

2

(S21/n1)2

n1+1 +

(S22/n2 )2

n2+1

2 t0 > t,

if1 = 2Amit (ISI, Chennai) ANOVA December 16, 2013 5 / 33
http://find/


30/163

Tests on variances of Normal Distributions


H0 : 2 = 20

20 >

2/2,n

1 or

H1 : 2 = 20

http://find/


31/163



H0 : 2 = 20

20 >

2/2,n

1 or

H1 : 2 = 20 20 <

21/2,n1

H0 : 2 = 20

2 =

(n1)S2

20

H1 : 2

< 20

http://find/


32/163



H0 : 2 = 20

20 >

2/2,n

1 or

H1 : 2 = 20 20 <

21/2,n1

H0 : 2 = 20

2 =

(n1)S2

20

H1 : 2

< 20

20 <

21,n1

H0 : 2 = 20

H1 : 2

> 20

http://find/


33/163



H0 : 2 = 20

20 >

2/2,n

1 or

H1 : 2 = 20 20 <

21/2,n1

H0 : 2 = 20

2 =

(n1)S2

20

H1 : 2

< 20

20 <

21,n1

H0 : 2 = 20

H1 : 2

> 20

20 >

2,n1

H0 : 21 =

22

F0 = S21

S22

H1 : 21 =

22

http://find/


34/163



H0 : 2 = 20

20 >

2/2,n

1 or

H1 : 2 = 20 20 < 21/2,n1H0 :

2 = 20

2 =

(n1)S2

20

H1 : 2

< 20

20 <

21,n1

H0 : 2 = 20

H1 : 2

> 20

20 >

2,n1

H0 : 21 =

22

F0 = S21

S22

H1 : 21 =

22 F0 > F/2,n11,n21

F0 < F1/2,n11,n21H0 :

21 =

22

F0 = S22

S21

H1 : 21 <

22

http://find/


35/163



H0 : 2 = 20

20 >

2/2,n

1 or

H1 : 2 = 20 20 < 21/2,n1H0 :

2 = 20

2 =

(n1)S2

20

H1 : 2

< 20

20 <

21,n1

H0 : 2 = 20

H1 : 2

> 20

20 >

2,n1

H0 : 21 =

22

F0 = S21

S22

H1 : 21 =

22 F0 > F/2,n11,n21

F0 < F1/2,n11,n21H0 :

21 =

22

F0 =

S22S21

H1 : 21 <

22 F0 > F,n21,n11

H0 : 21 =

22

F0 = S21

S22H1 :

21 >

22

http://find/


36/163



H0 : 2 = 20

20 >

2/2,n

1 or

H1 : 2 = 20 20 < 21/2,n1H0 :

2 = 20

2 =

(n1)S2

20

H1 : 2

< 20

20 <

21,n1

H0 : 2 = 20

H1 : 2

> 20

20 >

2,n1

H0 : 21 =

22

F0 = S21

S22

H1 : 21 =

22 F0 > F/2,n11,n21

F0 < F1/2,n11,n21H0 :

21 =

22

F0 =

S22S21

H1 : 21 <

22 F0 > F,n21,n11

H0 : 21 =

22

F0 = S21

S22H1 :

21 >

22 F0 > F,n11,n21


A k tt l
http://find/


37/163

A kutty example

Let a set of observations be 5.1, 5.3, 5.0, 5.2, understandably the sd is low!


A kutty example
http://find/


38/163

A kutty example


Similar is the sd in another data set, like 10.9, 10.6, 10.8, 10.7.


A kutty example
http://find/


39/163

A kutty example



So an average sd for the two sets of data taken together is also going tobe low!


A kutty example
http://find/


40/163

A kutty example




So how do we do this averaging ?


A kutty example
http://find/


41/163

A kutty example




So how do we do this averaging ?

Do you know the paveraging problem ?


do and dont
http://find/


42/163

do and don t

So, in one batch of production out of100 there were no defective,


do and dont
http://find/


43/163

do and don t

So, in one batch of production out of100 there were no defective, and

only one was produced in the next batch and


do and dont
http://find/


44/163

do and don t


only one was produced in the next batch and the piece unfortunately wasbad!


do and dont
http://find/


45/163

do and don t



So the data was as follows :


do and dont
http://find/


46/163

do and don t




No. produced no. defective Proportion100 0 0

1 1 1average of two ps 0.5

Can you believe that??


do and dont
http://find/


47/163

do a d do t




No. produced no. defective Proportion100 0 0

1 1 1average of two ps 0.5

Can you believe that??

However, the following is what you SHOULD do.

ratio of total def & tot produced 1101


Averaging SDs
http://find/


48/163

g g

1

So we could find a sd taking all the eight observations from the twosets. And thats going to be large.


Averaging SDs
http://find/


49/163

g g

1


2 However the average of the two sds calculated for each set separatelyis going to small.


Averaging SDs
http://find/


50/163

g g

1



The two above would be very different and a comparison for equality willfail.


Averaging SDs
http://find/


51/163

g g

1




Why is this?


Averaging SDs
http://find/


52/163

1




Why is this?

Because the two sets are different!


Averaging SDs
http://find/


53/163

1




Why is this?

Because the two sets are different!

But different in what?

http://find/


54/163

different : In the average level of course.

http://find/


55/163


This is why ANOVA works.

http://find/


56/163



ANOVA can be used to to test such hypothesis of equality of means due

to several factors simultaneously.

http://find/


57/163





Typically used to analyze data in a Design of Experiment (DOE) scenario.

http://find/


58/163





Typically used to analyze data in a Design of Experiment (DOE) scenario.

Though the method assumes normality of the underlying characteristics

and equality of variances of the populations are study, it is a very robustmethod, in the sense that minor deviations from the assumption do notinfluence the conclusions adversely.


Lets look at an Example now...
http://find/


59/163

http://find/


60/163

The tensile strength of synthetic fibre used to make cloth for mens shirt isof interest to a manufacturer. It is suspected that the strength is affected

by the % of cotton in the fibre.


http://find/


61/163


by the % of cotton in the fibre. Five levels of cotton % are of interest 15, 20, 25, 30 and 35.


http://find/


62/163

http://find/


63/163


by the % of cotton in the fibre. Five levels of cotton % are of interest 15, 20, 25, 30 and 35. Five observations are taken at each level of cotton%,and the 25 observations are run in random order. The following tablegives the values of Tensile Strength of Synthetic Fibre (lb/in2).


Example 1 continued...
http://find/


64/163

The data on Tensile strength of five repeat samples of yarn with differentcotton percentages are as in the table below.

% of Observationscotton 1 2 3 4 5

15 7 7 15 11 920 12 17 12 18 1825 14 18 18 19 1930 19 25 22 19 2335 7 10 11 15 11


http://find/


65/163

Some more initial computation of the data in the last table...


http://find/


66/163

Some more initial computation of the data in the last table...

% of cotton : Factor Observations(j) : yij TotalLevel(i) Value 1 2 3 4 5 yi.

1 15 7 7 15 11 9 492 20 12 17 12 18 18 773 25 14 18 18 19 19 884 30 19 25 22 19 23 1085 35 7 10 11 15 11 54

Total (G

=

i

jy

ij) 376


http://find/


67/163

In the following, we put the numbers in the table into the following

symbols.


http://find/


68/163


symbols..

yij=jth observation(Tensile Strength) on ith level of cotton %.


http://find/


69/163


symbols..


yi. = Total of the observations of ith level.

http://find/


70/163



71/163


symbols..



y.. = Total of all observations.

n= Number of observations per level


http://find/


72/163


symbols..





k= The number of levels we are comparing.


http://find/


73/163


symbols..





k= The number of levels we are comparing.

N= the total number of observations.


The ANOVA model
http://find/


74/163

In a One Way Analysis of Variance the hypothesis to be tested is that theitems in the various classes come from universes, the means of which areequal.


The ANOVA model
http://find/


75/163


More precisely, the mathematical model for the analysis is

yij=+i+ij

where is a constant, the is are the class differentials and ij is randomnormal deviate with mean zero and variance 2, these being the same forall classes.


The ANOVA model
http://find/


76/163



yij=+i+ij


The hypothesis to be tested is that the

is are zero for all j

.


The ANOVA model

O W f
http://find/


77/163

In a One Way Analysis of Variance the hypothesis to be tested is that theitems in the various classes come from universes, the means of which are

equal.


yij=+i+ij


The hypothesis to be tested is that the is are zero for all j.

With reference to the tensile strength example the null hypothesis means :the mean tensile strength of yarns with different cotton percentages are allequal.


The computations for a one way ANOVA
http://find/


78/163

One important result we need to know is that : means of samples of size nfrom a single population tends to have a variance that equals the varianceof the universe divided by n.

2y=2yn


http://find/


79/163


2y=2yn

Lets look at the formula for the standard deviation :


http://find/


80/163


2y=2yn

Lets look at the formula for the standard deviation :

The observations are : x1, x2, . . . , xn

2x= 1n 1

ni=1

x2i [n

i=1xi]2n


The computations
http://find/


81/163

1 The correction factor : CF =y2..

/N[

i

jyij]

2

N = 376

2

25 =


The computations
http://find/


82/163


/N[

i

jyij]

2

N = 376

2

25 =

2 Total sum of square SSTotal=

ijy2ij CF= (7)2 + (7)2 + (15)2 + + (15)2 CF= 636.96


The computations
http://find/


83/163


/N[

i

jyij]

2

N = 376

2

25 =


ijy2ij CF= (7)2 + (7)2 + (15)2 + + (15)2 CF= 636.963 Sum of squares due to cotton percentage SScotton =

i[y2i.]/n] CF =

15 [(49)

2 + (77)2 + (88)2 + (108)2 + (54)2] CF= 475.76


The computations


84/163


/N[

i

jyij]

2

N = 376

2

25 =



i[y2i.]/n] CF =

15 [(49)

2 + (77)2 + (88)2 + (108)2 + (54)2] CF= 475.76

4

Sum of squares due to errorSS

Error =SS

total SS

Cotton


The computations
http://find/


85/163


/N[

i

jyij]

2

N = 376

2

25 =



i[y2i.]/n] CF =

15 [(49)

2 + (77)2 + (88)2 + (108)2 + (54)2] CF= 475.76

4 Sum of squares due to error SSError

=SStotal

SSCotton

5 The above SSErrorcan also be calculated independently.


Degree of Freedom
http://find/


86/163

Degrees of freedom is the number of observations that are free to vary!


Degree of Freedom
http://find/


87/163


Well, its not easy, unfortunately to understand degree of freedom as a

concept in its entirety with little experience!


Degree of Freedom
http://find/


88/163


Well, its not easy, unfortunately to understand degree of freedom as a

concept in its entirety with little experience!

Do have a look athttp://www.creative-wisdom.com/computer/sas/df.html

Amit (ISI Chennai) ANOVA December 16 2013 18 / 33

Degree of Freedom

This is a deep and wide concept, not easy to get in a single go.
http://find/


89/163

http://find/


90/163

Degree of Freedom



91/163

However, here is an analogy! an analogy only not the whole concept.


Degree of Freedom

http://find/


92/163


This is a piece of chocolate I want to divide in three pieces.


Degree of Freedom

http://find/


93/163



For my friend!


Degree of Freedom


H h i l ! l l h h l
http://find/


94/163



For my friend!

A big piece for myself!


Degree of Freedom


H h i l ! l l h h l
http://find/


95/163



For my friend!


Do I have a choice for the last?


Degree of Freedom


H h i l ! l l t th h l t
http://find/


96/163



For my friend!


Do I have a choice for the last?

http://find/


97/163


98/163


99/163

Hypothesis Testing


100/163

H0 :1 =2 =4 =5H1 :i=j for some i=j

Conclusion1 : H0 rejected.

A it (ISI Ch i) ANOVA D b 16 2013 21 / 33

Hypothesis Testing

H
http://find/


101/163

H0 :1 =2 =4 =5H1 :i=j for some i=j


Further Analysis

A it (ISI Ch i) ANOVA D b 16 2013 21 / 33
http://find/


102/163

Hypothesis Testing

H


103/163

H0 :1 =2 =4 =5H1 :i=j for some i=j


Further Analysis

Level 1 2 3 4 5Avg.Response 9.8 15.4 17.6 21.6 10.8

Conclusion2

1 1 =5 < 2 =3< 4 and

A i (ISI Ch i) ANOVA D b 16 2013 21 / 33

Hypothesis Testing

H
http://find/


104/163

H0 :1 =2 =4 =5H1 :i=j for some i=j


Further Analysis

Level 1 2 3 4 5Avg.Response 9.8 15.4 17.6 21.6 10.8

Conclusion2

1 1 =5 < 2 =3< 4 and

2 95% confidence interval for 4 is 18.954 24.25


Terminologies

1 Factor
http://find/


105/163

http://find/


106/163

Terminologies

1 Factor


107/163

2 Level

3 Response

http://find/


108/163


109/163

Terminologies

1 Factor


110/163

2 Level

3 Response

4

Main effect

5 Interaction effect

6 One way ANOVA


Terminologies

1 Factor
http://find/


111/163

2 Level

3 Response

4

Main effect

5 Interaction effect

6 One way ANOVA

7 Two way ANOVA

http://find/


112/163

Example : Two-way

The maximum output voltage of a particular type of storage battery isthought to be influenced by the material used in the plates and thetemperature in the location at which the battery is installed.


113/163

temperature in the location at which the battery is installed.

http://find/


114/163

Example : Two-way

The maximum output voltage of a particular type of storage battery isthought to be influenced by the material used in the plates and thetemperature in the location at which the battery is installed.


115/163

p y

Four replicates of a factorial experiment are run in the laboratory for threetypes of material and three temperatures, and the results are presented asfollows.

Material Temperature(oF)Type 50 65 80

1 130 155 34 40 20 7074 180 80 75 82 58

2 150 188 136 122 25 70159 126 106 115 58 45

3 138 110 174 120 96 104168 160 150 139 82 60

http://find/


116/163


117/163


118/163


119/163

The two-way Model

yijk=+i+j+ijk


120/163

where is a constant, the is are the class differentials corresponding tothe row factor (in this example material type), jare the class differentialscorresponding to the column factor(temperature) and ijk is random normaldeviate with mean zero and variance 2, these being the same for all theclasses.

The hypotheses to be tested are that the is are zero for all i, js are zerofor all j.

With reference to the output voltage example the null hypothesis means :

the mean mean output voltage of batteries with different material type areall equal and also the mean output voltages for all material types are same.

i= 1, , 3. j= 1, , 3. k= 1, , 4.


Interaction Effect
http://find/


121/163

In an experiment with more than one factor there is another effect that weare concerned with, known as the interaction effect.

http://find/


122/163


123/163

Interaction demonstrated

Consider two factors A and B, both at two levels 1 and 2. Let theresponses be as follows :


124/163

http://find/


125/163




126/163

B1 B2A1 12 30A2 50 20

Pictorially the above scenario is as follows :




B B
http://find/


127/163

B1 B2A1 12 30A2 50 20


B1 B2




B B
http://find/


128/163

B1 B2A1 12 30A2 50 20


B1 B2

http://find/


129/163



B B


130/163

B1 B2A1 12 30A2 50 20


B1 B2

A1




B B
http://find/


131/163

B1 B2A1 12 30A2 50 20


B1 B2

A1

http://find/


132/163


133/163



B1 B2


134/163

B1 B2A1 12 30A2 50 20


B1 B2

A1

A2

http://find/


135/163


136/163

The Voltage Example

Correction Factor (CF) = (3799)2

36

Total Sum of Squares


137/163

Total Sum of Squares(SSTotal) = (130)

2 + (155)2 + (74)2 + + (82)2 + (60)2 CF= 77, 646.96

Sum of Squares due to Material(SSM) =

13x4

[(998)2 + (1300)2 + (1501)2] CF = 10, 683.72


The Voltage Example


36

http://find/


138/163

q(SSTotal) = (130)

2 + (155)2 + (74)2 + + (82)2 + (60)2 CF= 77, 646.96


13x4

[(998)2 + (1300)2 + (1501)2] CF = 10, 683.72

Sum of Squares due to Temperature(SST) =

13x4 [(1738)

2 + (1291)2 + (770)2] CF = 39, 118.72


The Voltage Example


36

http://find/


139/163

q(SSTotal) = (130)

2 + (155)2 + (74)2 + + (82)2 + (60)2 CF= 77, 646.96


13x4

[(998)2 + (1300)2 + (1501)2] CF = 10, 683.72


13x4 [(1738)

2 + (1291)2 + (770)2] CF = 39, 118.72

Sum of Squares due to Interaction of Material and TemperatureSSMxT =

14 [(539)

2 + (229)2 + + (342)2] CF SSM SST = 9, 613.77


The Voltage Example


36

Total Sum of Squares2 2 2 2 2
http://find/


140/163

q(SSTotal) = (130)

2 + (155)2 + (74)2 + + (82)2 + (60)2 CF= 77, 646.96


13x4

[(998)2 + (1300)2 + (1501)2] CF = 10, 683.72


13x4 [(1738)

2 + (1291)2 + (770)2] CF = 39, 118.72

Sum of Squares due to Interaction of Material and TemperatureSSMxT =

14 [(539)

2 + (229)2 + + (342)2] CF SSM SST = 9, 613.77

Sum of Squares due to Error SSError =SSTotal SSM SST SSMxT =77, 646.96 10, 683.72 39, 118.72 9, 613.77 = 18, 230.75.


The ANOVA table

SV DF SS MS F i
http://find/


141/163

SV DF SS MS F ratioMaterial 2 10,683.72 5,341.86 7.91

Temperature 2 39,118.72 19,55836 28.97

Interaction 4 9,613.77 2,403.44 3.56

Error 27 18,230.75 675.21Total 35 77,646.96

SV : Source of variation, DF : Degree of freedom,SS : Sum of squares, MS : Mean square.

http://find/


142/163

An Exercise

The effective life of a cutting tool installed in a numerically controlledmachine is thought to be affected by the cutting speed and the tool angle.

Three speeds and three angles are selected, and a factorial experiment


143/163

with two replicates is performed.

http://find/


144/163


145/163

Some questions...

How many factors?


146/163


Some questions...

How many factors?

How many levels in which factor?
http://find/


147/163

y

http://find/


148/163


149/163


150/163


151/163


152/163

Some questions...

How many factors?

How many levels in which factor?


153/163

Are there replications?

So what are all the sources of variations?

Do we know all the degrees of freedoms?

Can we write down the ANOVA Table?

Do we now know what the model should be?

Finally do we know what to compute and how?


Some questions. . .

What is pvalue?
http://find/


154/163

http://find/


155/163

Some questions. . .

What is pvalue?


156/163

What are Type 1 and Type 2 errors.

Level of significance?


Some questions. . .

What is pvalue?


157/163



?


Some questions. . .

What is pvalue?
http://find/


158/163



?

What are the purposes ofRandomization, Replication and Local control?

http://find/


159/163

A case

Growth of a particular infection ia being studied on several differentmedically important substrates.


160/163

Several drugs are also being studied for their efficacy against the growth.


A case

http://anova-output.pdf/http://find/


161/163


It is desired to investigate if the substrates behave differently with

reference to the growth.


A case



162/163



reference to the growth.

Also if some drugs are more effective than the other with reference to thetreatment.


A case



163/163



reference to the growth.Also if some drugs are more effective than the other with reference to thetreatment.

Lets look at a typical output of ANOVA from MINITAB. Click Here

http://anova-output.pdf/http://anova-output.pdf/http://find/

A Nova Presentation

Documents

Transcript of A Nova Presentation