1

Systems Management

Control Concepts

“It is the mark of a truly intelligent person to be moved by statistics.”

George Bernard Shaw

2

R. A. Fisher

• Studied math and astronomy at Cambridge

• Conducted agricultural research in Britain

• Developed powerful methods of experimental design

Sir Ronald A. Fisher1890 - 1962

Lady Tasting Tea• At a summer tea party in Cambridge, England, a

lady states that tea poured into milk has a different taste than milk poured into tea

• Most guests, including distinguished scientists, think this nonsense, but Ronald Fisher proposes a method to scientifically test the lady's claim

• In Fisher’s experiment, the lady correctly identified every cup!

Salsburg, D., The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century,” W.H. Freeman, 2001

A Cure for Scurvy• Scurvy once common among sailors

• Dr. James Lind, a Scottish surgeon in the British Royal Navy, designed experiments in 1754

– 12 scurvy patients on the HMS Salisbury selected

– Divided into six pairs and given remedies

• Cider, vitriol, seawater, garlic, vinegar, citrus

– The pair on oranges and lemons fit for duty in six days

• First controlled clinical trial

• Origin of “limey” as term for British sailors

Lind, J. , A Treatise of the Scurvy, 1753

5

Design of Experiments

• As we vary process parameters to reduce the loss function, it may be hard to determine what changes cause what effects

• Traditional method is to vary one parameter at a time– Inefficient– Provides no information about interactions

6

Example

• Goal: improve some quality characteristic

• Factors: temperature (T) and pressure (P)

• Approach: conduct a series of experiments– First, the classic one

factor at a time design

– Baseline plus 2 trials

Trial T P Result

1 L L 10

2 L H 30

3 H L 20

7

One Factor at a Time Design

• The “T-effect” increase result by 10

• The “P-effect” increase result by 20

• Each estimate based on two trials

• No estimate of the interaction effect

Trial T P Result

1 L L 10

2 L H 30

3 H L 20

8

Factorial Design

• The T-effect

(20+26)/2 – (10+30)/2 = 3• The P-effect

(30+26)/2 – (20+10)/2 = 13• Each estimate based on

four trials• Next, interaction effect

Trial T P R

1 L L 10

2 L H 30

3 H L 20

4 H H 26

9

Factorial DesignTxP Interaction Effect

• T-effect (P Low) 20 – 10 = 10

• T-effect (P High)26 – 30 = -4

• By convention TxP effect is half the difference

(10+4) / 2 = 7• Average of two T-effects

equal to main effect of T[10 + (-4)] / 2 = 3

Trial T P R

1 L L 10

2 L H 30

3 H L 20

4 H H 26

10

Factorial DesignInteraction Effect

0

5

10

15

20

25

30

35

T-Low T-High

P-Low P-High

T P R

1 L L 10

2 L H 30

3 H L 20

4 H H 26

Interaction: Effect of T changesdepending on level of P

Best result

11

Factorial DesignPxT Interaction Effect

• P-effect (T Low)

30 – 10 = 20

• P-effect (T High)

26 – 20 = 6

• Half the difference is the PxT effect (same as TxP)

(20-6) / 2 = 7

• Average of two P-effects equal to main effect of P

[20 + 6] / 2 = 13

Trial T P R

1 L L 10

2 L H 30

3 H L 20

4 H H 26

12

Factorial DesignLurking Variable

• Suppose Operator also affects the response

• T and O perfectly correlated

• Which variable caused the effect attributed to T?

• Control what you can

• Randomize trial order to break link with variables you can’t control (or don’t know about)

Trial T P R O

1 L L 10 1

2 L H 30 1

3 H L 20 2

4 H H 26 2

13

Factorial DesignSummary

• With only four trials, we get– Main effect of T– Main effect of P– 2-factor interaction effect

• Each estimate based on all four trails

• Estimating effects ANalysis Of Means (ANOM)

• Determining which effects are statistically significant ANalysis Of VAriances (ANOVA)

T P R

1 L L 10

2 L H 30

3 H L 20

4 H H 26

With good design, analysis can be simple

14

Factorial Design

• Same as earlier example

• Here with algebraic notation

• Recall that

T-effect = 3

P-effect = 13

T x P effect = 7

• Much easier to get these results with this notation

Trial T P TxP R

1 + 10

2 + 30

3 + 20

4 + + + 26

Calculating Effects in the 23 Design

A B C AB AC BC ABC

+ + + + + + +

+ + +

+ + +

+ + +

+ + +

+ + +

+ + +

+ + +

15

The Fractional Factorial

A B C AB AC BC ABC

+ + + + + + +

+ + +

+ + +

+ + +

16

• One-half fraction of a full 23 design: 23-1 design

• Note that effects B, AB and BC, ABC are confounded

• We always lose something with a fractional design

Fractional Factorial Designs

• As the number of factors increases, the runs required for a full factorial may outgrow the experimental resources available

• Fractional factorial designs are used for screening experiments: to determine which factors are truly significant

• Goal: distinguish main effects and 2-factor interactions

17

Alias structure for 26-2 designEffect Alias Alias Alias

A BCE CDF ABDEF

B ACE DEF ABCDF

C ABE ADF BCDEF

D ADF BEF ABCDE

E ABC BDF ACDEF

F ACD BDE ABCEF

AB CE BCDF ADEF

AC BE DF ABCDEF

AD CF BCDE ABEF

AE BC CDEF ABDF

AF CD BCEF ABDE

BD EF ACDE ABCF

BF DE ABCD ACEF 18

Design Resolution• Resolution III Designs (23-1)

– No main effects are aliased with any other main effect, but main effects are aliased with 2-factor interactions

• Resolution IV Designs (24-1)– No main effect is aliased with any other main effect or 2-

factor interaction, but 2-factor interactions are aliased with each other

• Resolution V Designs (25-1)– No main effect or 2-factor interaction is aliased with any

other main effect or 2-factor interaction

19

20

George Box

• Born in Britain

• Distinguished career at the University of Wisconsin

• Box, Hunter, and Hunter, Statistics for Experimenters

• Dr. John MacGregor one of Box’s Ph.D. students George E. P. Box

1919 -

21

Douglas Montgomery

• Ph.D. VPI, 1969

• G.T. faculty, 1969 - 84

• ASU faculty, 1988 – present

• ACQC Shewhart Medal, 1997

• Supervised Philip Coyle’s M.S. thesis, "An Adaptation of Bayesian Statistical Methods to the Determination of Optimal Sample Sizes for Operational Testing"

How to use statistical techniques

• Find out as much as you can about the problem• Get help from experts on the process• Define objectives• Don’t invest more than one quarter of the

experimental effort (budget) in a first design

• Box, G., Hunter, W., Hunter, S., Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, John Wiley and Sons, 1978

22

How to use statistical techniques

• Use your non-statistical knowledge of the problem• Keep the design and analysis as simple as possible• Recognize the difference between practical and

statistical significance• Experiments are usually iterative

• Montgomery, D., Design and Analysis of Experiments, 2nd edition, John Wiley and Sons, 1984

23