Better Industrial and Scientific Experiments: The Overview and New Directions
byJames M. Lucas and Derek F.
Webb2002 Fall Technical Conference
Valley Forge,PAOctober 17-18, 2002
J. M. Lucas and Associates 2
Contact InformationJames M. LucasJ. M. Lucas and
Associates5120 New Kent Road Wilmington, DE
19808(302) [email protected].
net
Derek F. WebbBemidji State Univ.HS-341, Box 231500 Birchmont Dr.
NEBemidji, MN 56601(218) [email protected]
J. M. Lucas and Associates 4
QUESTIONS How many of you are involved with
running experiments? How many of you “randomize” to guard
against trends or other unexpected events?
If the same level of a factor such as temperature is required on successive runs, how many of you set that factor to a neutral level and reset it?
J. M. Lucas and Associates 5
ADDITIONAL QUESTIONS
How many of you have conducted experiments on the same process on which you have implemented a Quality Control Procedure?
What did you find?
J. M. Lucas and Associates 6
COMPARING THE RESIDUAL STANDARD DEVIATION FROM AN EXPERIMENT WITH THE RESIDUAL STANDARD DEVIATION FROM AN IN-CONTROL PROCESS
MY OBSERVATIONS
EXPERIMENTAL STANDARD DEVIATION IS LARGER. 1.5X TO 3X IS COMMON.
J. M. Lucas and Associates 7
Research Team Huey Ju Jeetu Ganju Frank Anbari
Malcolm Hazel Derek Webb John Borkowski
J. M. Lucas and Associates 8
IMPLICATIONS
HOW SHOULD EXPERIMENTS BE CONDUCTED?
•“COMPLETE RANDOMIZATION” (and the completely randomized design)
•RANDOM RUN ORDER (Often Achieved When Complete Randomization is Assumed)
•SPLIT PLOT BLOCKING (Especially When There are Hard-to-Change Factors)
J. M. Lucas and Associates 9
Results for Experiments with Hard-to-Change and Easy-to-Change Factors
One H-T-C or E-T-C Factor: use split-plot blocking
Two H-T-C Factors: may split-plot Three or more H-T-C Factors:
consider RRO or Low Cost Options Consider “Diccon’s Rule”: Design
for the H-T-C Factor
J. M. Lucas and Associates 10
Not Resetting Factors Common practice Not addressed by the classical
definition Gives a split-plot blocking structure with
the blocks determined at random May be cost effective Causes biased hypothesis tests over all
randomizations (Ganju and Lucas 1997)
J. M. Lucas and Associates 11
RRO EXPERIMENTS (Random Run Order Without Resetting Factors)
OFTEN USED BY EXPERIMENTERS NEVER EXPLICITLY RECOMMENDED
ADVANTAGES•Often achieves successful results•Can be cost-effectiveDISADVANTAGES•Often can not be detected after experiment is conducted (Ganju and Lucas 99)•Biased tests of hypothesis (Ganju and Lucas 97, 02)•Can often be improved upon•Can miss significant control factors
J. M. Lucas and Associates 12
AN ESSENTIAL INGREDIENT OFRANDOM RUN ORDER (RRO)EXPERIMENTS (DuPont QM&TC)
J. M. Lucas and Associates 13
ADVANTAGES OF COMPLETE ADVANTAGES OF COMPLETE RANDOMIZATIONRANDOMIZATION
INDEPENDENT ERRORS GUARDS AGAINST TRENDS AND CYCLES VALIDATES RANDOMIZATION TESTS SIMPLE ANALYSIS NOTE: IN ADDITION TO A RANDOM RUN
ORDER, THE PHYSICAL ACT OF RESETTING IS NEEDED TO ACHIEVE “COMPLETE” RANDOMIZATION
J. M. Lucas and Associates 14
DISADVANTAGES OF COMPLETE DISADVANTAGES OF COMPLETE RANDOMIZATIONRANDOMIZATION
MORE TIME REQUIRED MORE EXPENSIVE LESS EFFICIENT
For easily changed factors
J. M. Lucas and Associates 16
A Fundamental TheoremTheorem 1. The expected covariance matrix, V, for an
experiment, which uses a random run order, is:
V p I p UUs w w ( ( ) ) ( ) 2 2 21
With one hard-to-change factor, the value of p is
• 1, for a completely restricted experiment;
• 0, for a completely randomized experiment;
• , for a random run order. 2)1(
21 LLk
Ju 1992, Ju and Lucas 2002, extended by Webb 1999, Webb, Lucas and Borkowski 2002
Some Examples of Super-Efficient Experiments
Optimum Blocking with one or two Hard-to-Change Factors
J. M. Lucas and Associates 18
24 with one Hard-to-Change Factor Obs. A B C D Blk 1 - - - - 1 2 - - + + 1 3 - + + - 1 4 - + - + 1 5 + - - - 2 6 + - + + 2 7 + + - + 2 8 + + + - 2
Obs. A B C D Blk 9 - - - + 3 10 - - + - 3 11 - + + + 3 12 - + - - 3 13 + - - + 4 14 + - + - 4 15 + + - - 4 16 + + + + 4
J. M. Lucas and Associates 19
24 with two Hard-to-Change Factors Obs. A B C D Blk. 1 - - - - 1 2 - - + + 1 3 - + + - 2 4 - + - + 2 5 + - - - 3 6 + - + + 3 7 + + - + 4 8 + + + - 4
Obs. A B C D Blk.
9 - - - + 5 10 - - + - 5 11 - + + + 6 12 - - - - 6 13 + - - + 7 14 + - + - 7 15 + + - - 8 16 + + + + 8
J. M. Lucas and Associates 20
24 Split-Plot is Super Efficient Main Effects plus interaction Model
11 Terms = (1 + 4 + 6) For I and A the variance is
02/16 + 1
2/4 For other terms it is 0
2/16 All terms of a CRD have (0
2 + 12 )/16
G-efficiency= 11(0
2 + 12)/(11 0
2 + 8 12)
J. M. Lucas and Associates 21
24 with two Hard-to-Change Factors
Nest Factor B within each A block giving a split-split-plot with 8 Blocks I=A1=BCD1=ABCD1=B2=AB2=CD2=ACD2
I and A have variance 02/16 + 1
2/4 +22 /8
B, AB and CD have 02/16 + 2
2 /8 Other terms have variance 0
2/16 G-efficiency =
11(02+1
2+22)/(110
2+812+102
2 ) >1.0
J. M. Lucas and Associates 22
2 BlocksBlock Size = 8
4 Blocks Block Size = 4
Lambdar = $Hard/$Easy is the Ratio of the Costs of Changing the Hard-to-Change Factor and the Easy-to-Change Factors.
LAMBDA is the Ratio of the Hard-to-Change Factor's Variance Component and the Other Variance Component.
IMPLICATIONSOptimum Block Size (Considering Costs)
24 DesignMain Effectsand 2 FactorInteractionsModel
r
J. M. Lucas and Associates 23
26-1 with one or two Hard-to-Change Factors
Main Effects plus interaction Model 22 Terms = (1 + 6 + 15)
Use Resolution V, not VI with I=ABCDEUse four blocks I=A=BCF=ABCF=BCDE=ADEF=DEF
Nest Factor B within each A block giving a split-split-plot with 8 Blocks =B2=AB2=CF2=ACF2=CDE2=ABDEF2=BDEF2
I and A have variance 02/32 + 1
2/4 +22 /8
B, AB and CF have 02/32 + 2
2 /8 Other terms have variance 0
2/32 G-efficiency =
22(02+1
2+22)/(220
2+1612+202
2 ) >1.0 Drop 2
2 terms for one h-t-c factor results
J. M. Lucas and Associates 25
24 Main Effects Model Super-Efficient 8 blocks design with
I=A=BC=ABC=CD=ACD=BD=ABD V(b0) = V(b1) = A
2/8 + 2/16
V(b2) = V( b3)= V( b4) = 2/16
max variance = (4A2 + 52)/16
Design can be improved upon when A
2/2 >2.5 by a 12-block design
J. M. Lucas and Associates 26
12-Block 24 Design I = CD in four blocks; I = -CD in eight blocks
Randomized four block weightsV(b0) = V(b1) (A
2 + 2)/8 A2/4 + 2/8 2/3, 1/3
V(bi,i>1) (A2 + 2)/8 2/8 0, 1
The maximum variance is: max variance = A
2/6 + 372/72
Super-Efficient max variance = (4A2 + 52)/16
A2=3, 2=1 gives 73/72 vs 17/16
J. M. Lucas and Associates 27
IMPLICATIONS SUPER EFFICIENT EXPERIMENTS (With One or Two Hard-to-Change Factor) SPLIT PLOT BLOCKING GIVES HIGHER PRECISION AND LOWER COSTS THAN COMPLETELY RANDOMIZED EXPERIMENTS
Random Run Order Experiments
See Webb, Lucas and Borkowski handout
Useful for RSM and for more than two H-t-C Factors
J. M. Lucas and Associates 31
2k Changing O-F-A-TThe lowest cost 2k experiment
FACTORS (Ranked in Increasing H-T-C Order)
Factor 1 2 3 4 … First Consecutive Same Sign 1 2 4 8 … Then Switch Signs Every 2 4 8 16 … Number of Changes 2k-1 2k-2 2k-3 2k-4 … - - - - … + - - - … + + - - … - + - - … - + + - … + + + - … + - + - … - - + - … - - + + …
J. M. Lucas and Associates 32
Other O-F-A-T 2k Designs More Balanced 2-2-3 Changes instead of 1-2-4 Least Expensive Way to Run Require large S/N Ratio
Top Related