DOE Evaluation Presentation

7/27/2019 DOE Evaluation Presentation

1/29

Selecting Effective Designs

Presented by: Larry Scott

Process TechnologiesNorthville, Michigan

248-347-1522

Welcome to:

DOE Evaluation:

Agenda

Intro to DOEOverall Strategy of DOEDesign Evaluation: Focus onFraction of Design Space (FDS) Full Factorial Designs Fractional Designs

CCD, Box Benken, D-Optimal, MLC Simplex Designs (Formulations)


2/29

ProcessProcess

Controllable Factors (X)Controllable Factors (X)

Responses (Y)Responses (Y)

Uncontrollable Variables (Z)Uncontrollable Variables (Z)

What is DOE:What is DOE: 66--sigma enthusiast Y = f (xsigma enthusiast Y = f (x ii))

DOE is:

A series of tests,

in which purposeful changes

are made to input factors,

so that you may identify causes

for significant changes

in the output responses.

HistoryFisher & Yates: DOE concepts 1920s

Plackett & Burman designs 1940s

1st Textbook: Box, Hunter & Hunter - 1960

Optimal Designs: a- , d- 1960s

Douglas Montgomery resurgence 1976

DOE software commercially available 1980s

Six Sigma drives interest in DOE 1990sMinimum Run Designs; Whitcomb & Oley 1999

Fraction of Design Space (FDS) Introduced - 2007


3/29

Phase: Screening

Designs: resolution IV 2k-p

Phase: Characterization

Designs: 2k, resolution V 2k-p

Phase: Optimization

Designs : CCD, BB, etc.

Phase: Verification

Designs: resolution III 2k-p

yes

Strategy of Experimentation

Factor effectsand interactions

ResponseSurfacemethods

Curvature?

Confirm?

KnownFactors

UnknownFactors

Screening

Backup

Celebrate!

no

no

yes

Trivialmany

Vital few

Presentation Intent

Applying design evaluationstechniques that ensure effective designproperties for YOUR experiments.

Concept of Power: an under utilizedtool in DOE, until NOW!until NOW!


4/29

Input Requirements for Design

Signal estimates

Noise (variance) estimates

Resolution: Main Effects & Interactions

Resource Availability

# of Model Coefficients

Signal / Noise

The golf ball represents the signal: .

The grass represents the noise: .

/ = 1/2

p = 8.6 %/ = 1

p = 19.5 %/ = 2

p = 57.2 %


5/29

System Signal

is reflected in the magnitude of the output.

is a function of the range or spread

of the input variables.

X1: Temperature - Low = 50 deg C

High = 100 deg C

X2: Pressure - Low = 10 psiHigh = 15 psi

Input Requirements for Design

Signal estimates

Noise (variance) estimates

Resolution: Main Effects & Interactions

Resource Availability

# of Model Coefficients


6/29


7/29

Selecting # of Design Points

Given how many factors (k) you want tostudy and the number of coefficients (p) inthe model you select, the design will bebuilt as follows:

Model: estimation of all coefficients.

Lack-of-Fit: test how well modelrepresents actual behavior.

Replicates: estimate pure error.

Why these inputs ?????

POWERPOWER !!!!!The ability to find a factor effect!


8/29

Power Depends On:

The size of the difference y being measured.the larger the difference the higher the power.

The size of the experimental error :the smaller the higher the power.

The risk chosen:the larger the higher the power.

The number of replicates:the more runs the higher the power.

Choose design appropriate to the problem:more orthogonal & larger designs have more power.

Agenda

Intro to DOE

Overall Strategy of DOE

Power using Fraction of Design Space

Full Factorial Designs: traditional (%)

Fractional Designs: FDS

CCD, Box Benken, D-Optimal, MLC

Simplex Designs (Formulations): FDS


9/29

Good Response Surface Designs

1. Allows chosen polynomial to be estimatedwell.

2. Sufficient information to test for lack of fit.

Have more unique design points thancoefficients in model.

Replicates to estimate pure error.

3. Remain insensitive to outliers, influential

values and bias from modelmisspecification.

4. Be robust to errors in the factor levels.

Good Response Surface Designs

5. Permit blocking and sequentialexperimentation.

6. Provide a check on varianceassumptions, e.g., test that residuals areN(0, 2).

7. Generate useful information throughout

the region of interest, i.e., provide aconstant distribution of variance acrossdesign space.

8. Do not contain an excessively largenumber of runs.


10/29

RSM vs OFATOFAT

-2 -1 0 1 2

30

45

60

75

90

Factor A

Response

-2 -1 0 1 2

60

65

70

75

80

85

90

Factor B

Response

Response

65

73

80

88

95

Response

-4-2

02

4

-4

-2

0

2

4

Factor A

Factor B

Response

LCD Case Study

The objective is to find the best

color/typeface combination to maximize

readability on a LCD video display terminal.

Response: Time (seconds)


11/29

1. Identify opportunity and define objective.Find best color/typeface combination tomaximize readability.

2. State objective in terms of measurableresponses.Response = Time in seconds.

a. Define the change (y) that is important todetect for each response. = 1 second

b. Estimate experimental error () for eachresponse. = 1 second.

c. Use the signal to noise ratio (//// = 1.0)to estimate power.

DOE Process: LCD Case Study

3. Enter the factor names, levels and units.

continue>>

Building the DesignLCD Case Study


12/29

4. Enter the response name, units, y (1), (1)and a / of 1 is calculated.

continue>>

Building the DesignLCD Case Study

Evaluating Power: LCD Case Study

Reading Time: = 1 sec, = 1 sec, //// = 1.0

Want power of at least 80% for effects ofinterest!


13/29

Power Depends On:

The size of the difference y being measured.the larger the difference the higher the power.

The size of the experimental error :the smaller the higher the power.

The risk chosen:the larger the higher the power.

The number of replicates:the more runs the higher the power.

Choose design appropriate to the problem:more orthogonal & larger designs have more power.

4.Build a 23 factorial with 2 replicates or 16runs.

Replicate to Increase Power:


14/29

5. Build a 23 factorial with 5 replicates or 40runs.

Replicate to Increase Power

However, As Collinearity Increases

Calculating power becomes a less

effective tool.. So..

We rely on a new tool..

Fraction of Design Space (FDS)

Plus:

StdErr: Prediction error for the design.


15/29

Stent Delivery System

A stent is a wire mesh tube used to prop open an

artery recently cleared using angioplasty. The stent

is collapsed to a small diameter over a balloon

catheter. It's then moved into the area of the

blockage.

When the balloon is inflated, the stent expands,

locks in place and forms a scaffold. This holds the

artery open. The stent stays in the arterypermanently, holding it open to improve blood flow

to the heart muscle.

Stent Delivery SystemThis case study is meant to illustrate typical DOE

use in research to develop an improved product; in

this case a stent delivery system. Typical factors

include:

Lengths and diameters of various components,

e.g. tip, balloon, catheter, etc.

Materials used for the components.

Assembly parameters, e.g. weld locations, howthe balloon is folded, etc.

Stent geometry, wall thickness, how it is

crimped on the balloon, etc.


16/29

Stent Delivery SystemMR-5 Factorial Design

Twelve factors (11 numeric and 1 categoric)were studied in a factorial design. Afteranalysis of the design:

A CCD design with seven factors wasrun.

The seven factors and the region of interest are:

(The actual factor names and levels are proprietary.)

Stent Delivery SystemMR-5 CCD Design

Factor Type Low Level()

High Level(+)

A numeric 1 +1

B numeric 1 +1

C numeric 1 +1

D numeric 1 +1

E numeric 1 +1

F numeric 1 +1

G numeric 1 +1


17/29

Stent Delivery SystemMR-5 CCD Design

Possible design choices:

27 CCD (143 builds and 152 runs)

27-1 CCD (79 builds and 88 runs)

MR-5 CCD (47 builds and 50 runs)

Small CCD (37 builds and 41 runs)

BB (57 builds and 62 runs)

Stent Delivery System

MR-5 CCD Design

Use Design-Expert to build a 7-factor MR-5 CCDdesign.


18/29

MR-5 DesignsProvide Considerable Savings

46610243010625614

3261024259225613232512218025612

212512206812811

192512195612810

17251218461289

1542561738648

1382561630647

1222561522326

MR52k-pkMR52k-pk

Check Size of Design

Confirm that the design has enough runs togive the results needed.

! "

This tool bar allows the userto estimate StdErr Mean for aspecific design based onthese inputs:


19/29

Check Size of Design

80% of the design space will be able to estimate theresponse with the needed precision.

# $

%&

0.00 0.25 0.50 0.75 1.00

0.000

0.475

0.950

1.425

1.899

#

Design-Expert Software

Min StdErr Mean: 0.302Max StdErr Mean: 1.899Cuboidalradius = 1Points = 10000t(0.05/2,14) = 2.14479Reference X = 0.80Reference Y = 0.619

FDS Graph

Fraction of Design Space

StdErrMea

0.00 0.25 0.50 0.75 1.00

0.000

0.475

0.950

1.425

1.899

Software Screenshot

# $

%&


20/29

Remember, As Collinearity Increases

Power calculations becomes a less

effective tool..

So, in Simplex designs we rely

extensively on ..

Fraction of Design Space (FDS)

calculations.

Simplex-Lattice Designs

X = 11

X = 13X = 12

{3, 3} Simplex Lattice {3,3} Simplex Latticeaugmented & 4 reps

,

X = 11

X = 13X = 12


21/29

Compare to Orthogonal Designs:

2

3

factorial with 3 level factorialcenter point

Design Evaluation

Two Simplex-Lattice Designs

X = 11

X = 13X = 12

{3, 3} Simplex Lattice {3,3} Simplex Latticeaugmented & 4 reps

,

X = 11

X = 13X = 12


22/29

Design Matrix Evaluation for MixtureSpecial Cubic Model

No aliases found

Degrees of Freedom for Evaluation

Model 6

Residuals 3

Lack 0f Fit 3

Pure Error 0

Corr Total 9

Simplex-LatticeWithout Augmentation or Replicates

&

Design Matrix Evaluation for MixtureSpecial Cubic Model

No aliases found

Degrees of Freedom for Evaluation

Model 6

Residuals 10Lack 0f Fit 6

Pure Error 4

Corr Total 16

Simplex-Lattice

With Augmentation and 4 Replicates

'


23/29

Term 0.5 Std. Dev. 1 Std. Dev. 2 Std. Dev.

A 6.2 % 9.8 % 23.8 %

B 6.2 % 9.8 % 23.8 %

C 6.2 % 9.8 % 23.8 %

AB 6.2 % 10.0 % 24.5 %

AC 6.2 % 10.0 % 24.5 %

BC 6.2 % 10.0 % 24.5 %

ABC 6.0 % 9.2 % 21.6 %

Simplex-LatticeWithout Augmentation or Replicates

( ! ) * $+

A 8.6 % 19.9 % 60.1 %

B 8.6 % 19.9 % 60.1 %

C 8.6 % 19.9 % 60.1 %

AB 7.6 % 15.6 % 46.7 %

AC 7.6 % 15.6 % 46.7 %

BC 7.6 % 15.6 % 46.7 %

ABC 7.8 % 16.5 % 49.6 %

Simplex-Lattice

With Augmentation and 4 Replicates

( ! ) * $+

, ) ) - )


24/29


FDS:

Calculates the volume of the design spacehaving a standard error (StdErr) less than orequal to a specified value.

The ratio of this volume to the total volume ofthe design volume is the fraction of designspace.

Produces a single plot showing the cumulative

fraction of the design space on the x-axis(from zero to one) versus the StdErr on the y-axis.

Simplex-Lattice

Without Augmentation or Replicates

A: A1.00

B: B

1.00

C: C

1.00

0.00 0.00

0.00

0.63 0.63

0.63

0.780.78

0.78

0.78

0.78 0.78

0.78

StdErrMean

0.00 0.25 0.50 0.75 1.00

0.00

0.25

0.50

0.75

1.00

0.63

0.78

Fraction of Design Space% $


25/29

Simplex-LatticeWith Augmentation and 4 Replicates

A: A1.00

B: B1.00

C: C1.00

0.00 0.00

0.00

StdErr of Design

0.470.47

0.47

0.570.57

0.57

0.57 0.57

0.57

0.572

2

2 2


StdErrMean

0.00 0.25 0.50 0.75 1.00

0.00

0.25

0.50

0.75

1.00

0.47

0.57

% $

Simplex-Lattice: 3DWithout With

A (1.000)

B (0.000)

C (1.000)C (0.000)

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

StdErrofDesign

A (0.000)

B (1.000)

StdErr

A (1.000)

B (0.000)

C (1.000)C (0.000)

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

StdErrofDesign

A (0.000)

B (1.000)

StdErr


26/29

Evaluating Mixture Designs

All mixture DOEs have collinearity due to the equalityconstraint, ie. proportions.. X1 + X2 + X3 = 100%

As other constraints are added collinearity worsens.The fall out is:

Power depend on orthogonality and thereforelooses value as collinearity increases.

Standard error plots and FDS are better design

evaluation tools in the presence of collinearity.

A : T E A - L S

28 .00

B : C o c a m i d e9 .00

C: Lauramide9 .00

1 .0 0 1 .0 0

20 .00

Height

140

150

160

160

160

170

176

22

22

22

22

22

P re d ic ti 1 7 6 .9 39 5 % L o 1 6 1 .9 29 5 % Hi 1 9 1 .9 4S E m e a 2 .8 0 65 4S E p re d 6 .8 8 8X 1 2 4 .4 7X 2 3 .4 9X 3 2 .0 4

Simplex: Simple Constraints


27/29

90

50

70

30

10

90

50

70

30

10

90

50

70

30

10

X1

X2

X3

1. 0.1 A

2. A 0.5

3.0

.1B

4.B

0.7

5. C 0.7

Simplex: Multi-component Constraints

90

50

70

30

10

90

50

70

30

10

90

50

70

30

10

X1

X2

X3

1

2

3

4

5

8

6

7

Additional Extreme Vertices


28/29

DOE Software

.(

-/

1) Readily generate many design options

2) Quick, powerful data analysis

3) Generate useful response surface models

4) Design evaluation include:

Power: percentage, contour & FDS plotsFDS plots

StdErr estimates for:

Mean optimization

Point prediction

Estimating differences

If you always do what you always did;

youll always get what you always got.

- wise but unknown philosopher

Old Habits Methods Die Hard


29/29

www.doetraining.com

Software: Design-Expertfrom Stat-Ease, Inc.

0)( ! 1

DOE Evaluation Presentation

Documents

Transcript of DOE Evaluation Presentation