Sample Size DeterminationText, Section 3-7, pg. 101

• FAQ in designed experiments (what’s the number of replicates to run?)

• Answer depends on lots of things; including what type of experiment is being contemplated, how it will be conducted, resources, and desired sensitivity

• Sensitivity refers to the difference in means that the experimenter wishes to detect

• Generally, increasing the number of replications increases the sensitivity or it makes it easier to detect small differences in means

• Choice of sample size is closely related to the probability of type II error .

• Hypotheses

Ho: 1 = 2

H1: 1 2

• Type I error – reject H0 when it is true ()

• Type II error – fail to reject H0 when it is false ()

• Power = 1 – P(Reject HoHo is false)

P(Fo > F,a-1,N-a Ho is false)

• The probability of type II error depends on the true difference in means = 1 - 2

Operating Characteristic Curves forFixed Effects/Equal Sample Sizes per Treatment Case

Operating characteristic curves plot against a parameter where

is related to . It depends on (0.01 and 0.05) and degrees of freedom for numerator (a-1) and denominator (N-a).

Assumptions and trials are needed to use the curves

2

2 12

a

ii

n

a

can be estimated through prior experience/ previous experiment/preliminary test/judgment, or assuming a range of likely values of .

• ith treatment effect

where

ssumed i’s can be used for which we would like to reject the null hypothesis with high probability

Example (etch rate experiment)

If the experimenter is interested in rejecting the null hypothesis with a probability of at least 0.90 if the five treatment means are

1 = 575 2 = 600 3 = 650 4 = 675

ii

a

iia 1

1

is planned

a = 4, N = an = 4n, a – 1 = 3, N – a = 4(n-1)

is calculated using assumed i’s

is assumed no larger than 25

Find the right plot:

a – 1 = 3 (= v1) determines the use of the upper plot on page 614 (Appendix Chart V)

Because the curves on the right side are used

62505

1

2 i i

nn

a

ni i 5.2

)25(4

)6250(22

5

1

22

Chart V. Operating characteristic curves for the fixed effects model analysis of variance (Page 614) the upper graph with v1=3 should be used.

The objective is to find ato see if the power is satisfied

It needs v2 (or n) to determine the particular curve, and a value of to determine

Therefore, at least n = 4

n a(n-1) Power (1- )

3 7.5 2.74 8 0.25 0.75

4 10.0 3.16 12 0.04 0.96

5 12.5 3.54 16 <0.01 >0.99

• It is often difficult to select a set of treatment means for choosing the sample size

• A very common way to use these charts is to define a difference in two means D of interest, then the minimum value of 2 is

• Typically work in term of the ratio of D/ and try values of n until the desired power is achieved

22

22

nD

a

Other Methods of Determining Sample Sizes

Specifying a Standard Deviation Increase• As the difference between means increase, the

standard deviation increases

• Choose a percentage P for the increase in of an observation beyond which the null hypothesis is rejected, equivalently

a

ii a

1

22 /

100/1

/1

22

P

aa

ii

Specifying a Standard Deviation Increase (continued)• Rearrange it,

• Therefore, can be expressed as

• Specify P, , and the probability to reject the null hypothesis, then determine n.

1100/1

/21

2

P

aa

ii

)(1100/1/

/21

2

nPn

aa

ii

Confidence Interval Estimation Method• Specify in advance how wide the confidence

intervals should be by specifying the accuracy of the confidence interval

• No OC curves are needed. Example: the etch rate experiment.

• Need and N (an) to determine t/2,N-a, and to estimate MSE

• Specify the level of confidence (95%, or =0.05), difference in mean to be determined (30Å/min), and (prior) estimate 2 (252 =625)

n

MSt E

aN

2,2/

Confidence Interval Estimation Method (continued)• Procedure: compare the accuracy with an assumed n,

with the specified accuracy (30Å/min)• When n = 5, t/2,N-a =2.120,

• When n = 6, t/2,N-a =2.086,

• When n = 7, t/2,N-a =2.064,

00.3052.335

)625(2120.2

2,2/ n

MSt E

aN

00.3011.306

)625(2086.2

00.3058.277

)625(2064.2

Dispersion Effects

• Focus is location effects so far using ANOVA: factor level means and their differences

• It needs constant variances. If not, using transformations to stabilize the variances.

• Sometime the dispersion effects are of interest: whether the different factor levels affect variability

• In such analysis, standard deviation, variance, or other measures of variability are used as response variables

An Example – Al Smelting Experiment

• A reaction cell: Alumina and other ingredients (with a certain ratio) under electric resistance heating

• Four different ratio control algorithms• Cell voltage is recorded (thousands of voltage

measurements during each run)• A run consists of one ratio control algorithm• Average cell voltage (affecting cell temperature), and

the standard deviation of cell voltage over a run (affecting overall cell efficiency) are response variables

• An ANOVA determines that the ratio control algorithm had no location effects (the ratio control algorithm does not change the average cell voltage)

• A transformation is used to study the dispersion effects

y = -ln(s)• A standard ANOVA can be done on y, the natural

logarithm of standard deviation -> algorithm 3 produces different standard deviation than others

1 2 3 4 5 6

1 4.93(0.05) 4.86(0.04) 4.75(0.05) 4.95(0.06) 4.79(0.03) 4.88(0.05)

2 4.85(0.04) 4.91(0.02) 4.79(0.03) 4.85(0.05) 4.75(0.03) 4.85(0.02)

3 4.83(0.09) 4.88(0.13) 4.90(0.11) 4.75(0.15) 4.82(0.08) 4.90(0.12)

4 4.89(0.03) 4.77(0.04) 4.94(0.05) 4.86(0.05) 4.79(0.03) 4.76(0.02)

ObservationsRatioControl

Algorithm