Where we’ve been & where we’re going

We can use data to address following questions:

1. Question: Is a mean = some number?Large sample z-test and CISmall sample t-test and CI

2. Question: Is a proportion = some %?Proportion version of large sample z-test and CI

Where we’ve been & where we’re going

3. Question: Is a diff between two means = some #

Independent samples:

large sample z test and CI

small sample t test and CI

paired samples:

small sample paired t test and CI

4. Question: Is diff between 2 proportions = some %

Proportion version of large sample z test and CI

Topics to be covered in remaining 8 classes (including today)

Analysis of Variance and Linear Regression (Chapters 11, 12 and 13)

“response = 0 + 1 covariate 1 + … +p covariate p + error”

Categorical Data / Contingency Tables“when response is discrete…”

4321

18

17

16

15

14

13

12

11

10

9

Fabric

Bur

n T

ime

Back to Fabric Data:Tried to light 4 samples of 4 different (unoccupied!) pajama fabrics on fire.

Higher #meanslessflamable

Mean=16.85std dev=0.94

Mean=10.95std dev=1.237 Mean=10.50

std dev=1.137

Mean=11.00std dev=1.299

Suppose we want to test:

H0:

HA: at least one mean is not equal.

at level = 0.05.

Note that this is the probability ofmaking a false claim (if they are all equal).

First idea for how to do this: do four tests at level ( etc and reject H0 if at least one is rejected.

Test 1H0:

HA: not equalLevel

Test 2

H0:

HA: not equalLevel

Test 3

H0:

HA: not equalLevel

Test 4

H0:

HA: not equalLevel

Reject all means equal if at least one test fails.

This will give you a decision, but what’s the overall probability of making a false claim (if all means are equal) (level) for this procedure?

>,<, or equal to ?

Overall =

Pr(Falsely reject H0: )

=Pr( at least one test falsely rejects)

=1-Pr(none falsely reject)

=1-Pr( test 1 doesn’t and … and test 4 doesn’t)

=1-(0.95^4) = 0.19(last step uses independence…)

Point: We thought we were doing a level 0.05 test, but it’s actually level 0.18! That’s a problem!

Name for this problem: multiple testing problem.What’s one solution?

Solution 1:

• Do the 4 tests each at a level less than • Many methods to do this: Bonferroni and

Tukey are some common ones.• We won’t go into much mathematical

detail, but these test are often conservative. (True is smaller than the planned and power is lower than planned.)

• For instance, divide by # of tests you do:1-(1-(/4))4 = 1-(1-0.05/4)4 = 0.0491…

Solution 2: Analysis of Variance!

Idea:

1. Variability in the fabric data occurs at two levels: within fabric type and across fabric types.

2. If across fabric type variability is “large” relative to variability within each fabric type, then the means are not equal.

4321

18

17

16

15

14

13

12

11

10

9

Fabric

Bur

n T

ime

Vertical spread of data points within each oval is one type of variability.

Vertical spread of the ovals is another type of variability.

• Suppose 12 = 2

2

• If 12 is estimated by s1

2 from n1 data points and 2

2 is estimated with s22 from n2 data

points (and the data are normal and independent), then

s22 / s1

2 ~ Fn2-1,n1-1

To use the idea to test, we need a fact about variances:

Another distribution.The F distribution.n2-1 = numerator dfn1-1 = denominator df

Use the test to define “large”

• H0:

• HA:

• Level test:

reject H0 at level if

s22 / s1

2 > F1-,n2-1,n1-1

• Test for fabric:Formally:– At least one of the means is different if:– Variance among fabric types is greater than

the variance within fabric types– Variance among fabric types / Variance within

fabric types > F1-,3-1,16-3

When one does the test, one uses software that produce:

Analysis of variance or ANOVA tables.

Suppose there are k treatments and n data points.ANOVA table:

Source Sum of Meanof Variation df Squares Square F P

Treatment k-1 SST MST=SST/(k-1) MST/MSE

Error n-k SSE MSE=SSE/(n-k)

Total n-1 total SS

ESTIMATE OF “WITHIN FABRIC TYPE” VARIABILITY

ESTIMATE OF “AMONG FABRIC TYPE” VARIABILITY

“SUM OF SQUARES” IS WHAT GOES INTO NUMERATOR OF s2: “(X1-X)2 + … + (Xn-X)2”

P-VALUEFOR TEST.(REJECT IF LESS THAN )

One-way ANOVA: Burn Time versus FabricAnalysis of Variance for Burn TimeSource DF SS MS F PFabric 3 109.81 36.60 27.15 0.000Error 12 16.18 1.35Total 15 125.99

Explaining why ANOVA is an analysis of variance:MST = 109.81 / 3 = 36.60Sqrt(MST) describes standard deviation among the fabrics.

MSE = 16.18 / 12 = 1.35Sqrt(MSE) describes standard deviation of burn time within each fabric type. (MSE is estimate of variance of each burn time.)

F = MST / MSE = 27.15It makes sense that this is large and p-value = Pr(F4-1,16-4 > 27.15) = 0 is small because the variance “among treatments” is much larger than variance within the units that get each treatment.

(Note that the F test assumes the burn times are independent and normal with the same variance.)