Design expert 9 tutorials 2015

Software, Training & Consulting: Statistics Made Easy® Rev. 5/2/14

Getting started with v9 of Design-Expert software 1

Design-Expert® Software: Why Version 9 is Mighty Fine!

What’s in it for You

Stat-Ease, Inc. welcomes you to version 9 (v9) of Design-Expert software (DX9) for design of experiments (DOE). Use this Windows®-based program to optimize your product or process. It provides many powerful statistical tools, such as:

Two-level factorial screening designs: Identify the vital factors that affect your process or product so you can make breakthrough improvements.

General factorial studies: Discover the best combination of categorical factors, such as source versus type of raw material supply.

Response surface methods (RSM): Find the optimal process settings to achieve peak performance.

Mixture design techniques: Discover the ideal recipe for your product formulation. Combinations of process factors, mixture components, and categorical factors: Mix your

cake (with different ingredients) and bake it too!

Your Design-Expert program offers rotatable 3D plots to easily view response surfaces from all angles. Use your mouse to set flags and explore the contours on interactive 2D graphs. Our numerical optimization function finds maximum desirability for dozens of responses simultaneously!

You’ll find a wealth of statistical details within the program itself via various Help screens. Take advantage of this information gold-mine that is literally at your fingertips. Also, do not overlook the helpful annotations provided on all reports.

For a helpful collection of checklists and ‘cheat sheets,’ see the Handbook for Experimenters. It’s free to all registered users. Furthermore, for quick primers on the principles of design and analysis, we recommend you read the following two soft-cover books from Stat-Ease Principals Mark Anderson and Pat Whitcomb —published by Productivity Press of New York city:

DOE Simplified: Practical Tools for Effective Experimentation, RSM Simplified: Optimizing Processes Using Response Surface Methods for Design of

Experiments.

Anderson and Whitcomb have also written a Primer on Mixture Design. It’s posted free for all to read via the “I’m a Formulator” link on the Stat-Ease home page.

Go to http://www.statease.com/prodbook.html for details and ordering information on the books listed above.

What’s New

Those of you who’ve used previous versions of Design-Expert software will be impressed with the many improvements in Version 9. Here are the highlights:

Hard-to-change factors handled via split plots Two-level, general and optimal factorial split-plot designs: Make it far easier as a practical

matter to experiment when some factors cannot be easily randomized.

http://www.statease.com/prodbook.html



Half-normal selection of effects from split-plot experiments with test matrices that are balanced and orthogonal: The vital effects, both whole-plot (created for the hard-to-change factors) and sub-plot (factors that can be run in random order), become apparent at a glance!

Effects from split plots assessed via REML* and

Kenward-Roger’s approximate F test: See the familiar p-values that tell you what’s statistically significant. *(Restricted maximum likelihood)

Design resolution provided for two-level factorial split plots: Assess from the start whether your choice suffices for screening main effects (Res IV) or characterizing interactions (Res V).

Power calculated for split plots versus the alternative of complete randomization: See how accommodation of hard-to-change factors degrades the ability to detect certain effects.

Check designs with restricted randomization for REML/OLS* equivalence: Keep things simple statistically (KISS) in the ANOVA. *(Ordinary least squares)

Other new design capabilities Definitive screening designs: If you want to cull out the vital few from many numeric process

factors, this fractional three-level DOE choice resolves main effects clear of any two-factor interactions and squared terms (see screen shot of correlation matrix—more on that later).

On the Factorial tab select a simple-sample design for mean-model only: Take advantage of powerful features in Design-Expert software for data characterization, diagnostics and graphics—for example with raw outputs from a process being run at steady-state.



Much-improved capabilities to confirm or verify model predictions New Post Analysis Node (at bottom of the handy tree structuring of Design, Analysis and

Optimization) contains Point Prediction, Confirmation and Coefficients Table reports: Old and new features gathered in logical place at the end of the natural progression from design through analysis.

Entry fields for confirmation data and calculation of mean results: Makes it really easy to see if follow-up runs fall within the sample-size-adjusted prediction intervals.

Enter verification runs embedded within blocks as controls or appended to your completed

design: Lend veracity to your ultimate model by these internal checks.

Verification points displayed on model graphs and raw residual diagnostics: See how closely these agree to what’s predicted by your model.

New and more-informative graphics Adjustably-tuned LOESS fit line for Graph Columns: Draw a curve through a non-linear set of

points as you see fit. *(Locally weighted scatterplot smoothing.)



Color-coded correlation grid for graph columns: Identify at a glance any factors that are not controlled independently of each other, that is, orthogonally; also useful for seeing how one response correlates to another.* *(Data shown in screen shot comes from historical data detailed in RSM Simplified on NFL sacks versus attributes of defensive linemen.)

Jump to run added to Factors Tool for model graphs:

For multidimensional experimental regions, find the slice of interest (containing the point you want to see) at the press of a button.

When jumping to a run, the range expands to include the design point: Use this feature to

check how well your model fits—comparing the actual result versus what is predicted via the surface graph (in this case very well—the circled red point is barely beneath the surface!).

Ignored (and missing) runs can be shown on graphs: Good to be reminded that the original

design called for this, but for one reason or another, you ignored the outcome (or the response could not be collected, or it was skipped).

Choice to do diagnostic graphs with externally-studentized residuals (now the default): This

deletion-diagnostic (vs internally-studentized) provides a more sensitive view of potential abnormalities.

B: Trip (mm)

D: Fast Shot (mm)

390 395 400 405 410

De

fects

(F

ractio

n)

0.00

0.20

0.40

0.60

0.80

1.00

D-

D+

Interaction



0.00 5.11 10.22 15.33 20.44

0.0

10.0

20.0

30.0

50.0

70.0

80.0

90.0

95.0

Half-Normal Plot

|Normal Effect|

Ha

lf-N

orm

al %

Pro

ba

bility

A-Homework

Many new icons, such as ones for Clear Points and Pop-Out View on the Diagnostics Tool: Jump to features used frequently more quickly via these handy markers (also they look good!).

Three-component contour graphs in reals: Get a better view of the restrictions placed on your

mixture space by the constraints you enter on each component and the total.

Half-normal plot for one-factor categorical experiments with replicates: See at a glance if anything significant emerges.

Greater flexibility in data display and export Descending sort of all individual design layout columns via right-click menu (shown) or

double-click on header (toggles with ascending sort—previously the only option): Helpful, for example, when minimum response is desired.

Identify via “Build Type” the predetermined Model, Lack of Fit, Center, and Replicate points in your design layout: Dissect the matrix laid out for optimal (I, D, etc) experiments.

Switch directly between continuous and discrete point type: Sometimes the settings for a

factor cannot be easily changed (for example, diameter of molded part)—then it pays to recognize them as discrete, thus enabling the numeric optimizer being set so it will not stray away from specific values.

Ignorable block and/or factor columns: Handy for “what-if” analysis, such as what would have happened if you had not blocked your experiment.

A: Water (%)5.000

B: Alcohol (%)

4.000

C: Urea (%)

4.000

2.000 2.000

3.000

Viscosity (mPa-sec)

40

60

60

8080

100

120

2

2 2

2



Journal feature to export data directly to Microsoft Word or Powerpoint: Fast and formatted for you to quickly generate a presentable report on your experimental results.

Improved copy/paste of Final Equation from the analysis of variance (ANOVA) report to

Microsoft Excel: This not only saves tedious transcription of coefficients but it also sets up a calculator for you to ‘plug and chug’, that is, enter into the spreadsheet cells what values for the inputs you’d like to evaluate and see what the model predicts for your response.

From Evaluation and ANOVA screens, the X matrix can be viewed and exported: This is helpful, for example, for copy and paste to R or Matlab where statisticians can do further manipulations for research purposes.

Display full precision of F-test: If just presenting p<0.0001 is not precise enough, show all the

decimals. New XML* script commands for exporting point predictions: Helpful for situations where one

wants to automate the transfer of vital outputs from Design-Expert to other programs. *(Extensible Markup Language)

More powerful tools for modeling Design model included in Fit Summary: This can be very helpful for combined designs such as

response surface optimizations that include categorical factors (in this case recommending a model that included some cross-product terms of 3rd order, which provided a better fit of the data).



All-hierarchical model (AHM) selection: Sort through all possible models up to the one you designed the experiment for, but all the while maintain hierarchy of terms so you do not end up with something ill-formulated. (PS. The alpha out is enforced after AHM is completed by doing a final sweep using backward selection, after which hierarchy is again corrected by the program.)

Non-linear equations involving trigonometric, exponential and other functions allowed for creating deterministic responses (for example—costs) or simulations: This will be especially helpful for setting up more realistic scenarios for students to solve during hands-on workshops for teaching DOE. (PS. Simulator now provides an entry field for ratio of variance between whole and sub plots so trainers can set up split-plot exercises.)

Special quartic Scheffé polynomial included in automatic selection for mixture modeling:

Sometimes this added degree (4th!) of non-linear blending helps to better shape the response surface—making it better for predictive purposes.

More choices when custom-designing your experiment Required model points set aside from optional additional ones that may be needed for

adequate sizing of the design: Prevents setting up an experiment with too few points to fit the chosen model.

Enter a single factor constraint for response surface designs: Creates a ‘hard’ limit on inputs that cannot go beyond a certain point (such as zero time) physically or operationally.



Greater flexibility in setting up models: For example you can now create an optimal model for experiment on mixtures with varying categorical ingredients, some of which can go to zero.* *(See presentation of “Categoric Mixture Components Proportion Going to Zero” by Pat Whitcomb, ENBIS-12, Ljubljana, Slovenia. Slides available on request to [email protected].)

Save candidate sets in actuals: More flexibility for customizing your experiment design.

More capability for numerical optimization Include Cpk* as a goal: Meet quality goals explicitly.

*(A process capability index widely used for Six Sigma and Design for Six Sigma programs.)

Enhanced design evaluation Random model generator provided to generate a realistic response via a quadratic

polynomial with coefficients picked by chance: Use this to play around with how the software presents the analysis—better than just generating random numbers that only fit a mean model.

One-sided option added to FDS* graph: Size your design properly for a verification

experiment done to create a QBD** design space. *(Fraction of design space) **(Quality by Design—a protocol promoted by the US Food & Drug Administration (FDA).)

Many things made nicer, easier, more configurable and faster Components that do not vary in a mixture experiment can now be included in the design

build—see them highlighted with gray in the layout: Provide a recipe sheet that encompasses the entire formulation, not just what will be manipulated in your study.

Automatically re-sort by run order after re-randomizing: A little feature that saves users a

bother. Diagnostics report now can be sorted by any of the statistics listed: This enables a more

informative ordering than by run number (the default). Faster display of graphs: Great for dazzling your audience with 3D graphs in high resolution.

mailto:[email protected]



Pop-out views numbered: Makes it easier to distinguish and find the associated view-Tool when re-arranging on your desktop.

Graph state stored with file: Restores setting to the way you liked them. Improved graphics on Transformation screen: Looks more elegant—better to show off your

results via live presentations or webinars.

Fonts on analysis tabs now configurable under Edit Preferences (Dialog Control): Go ahead and make them Comic Sans if you would like to lighten things up. ; )

Safety net expanded—more mistakes caught and ‘heads-ups’ given Warning when largest effect not selected on half-normal plot: This would not make sense, but

it might happen due to, for example, not lassoing points correctly.

Hover Help added to select fields: When your mouse goes over an entry place, the program fills you in with a bit more information on what’s entailed in the feature you are specifying.

0.00 0.10 0.20 0.30 0.40 0.50

0

10

20

30

50

70

80

90

95

99

Half-Normal Plot

|Standardized Effect|

Ha

lf-N

orm

al %

Pro

ba

bili

ty

B-Flow

D-Mud

Warning! Largest effect not selected.



Niceties that only statisticians might truly appreciate Mean correction for transformation bias when responses displayed in original scale: All you

need to know is that our statisticians figured out how to eliminate a tricky, little-known bias! Propagation of error (POE) carried out to the second derivative: Makes POE more accurate.

Display confidence bands with or without POE added: Easier to match output with other programs that do not offer POE features like this.

Add unblocked results to evaluation of blocked experiments: Aids in comparing designs on

the basis of matrix measures.

Scale to largest estimable effect those normal effects that cannot be otherwise estimated: This can happen when effects become too large compared to the error estimated by chi-square.

Preference now available to display p values to full precision: Previously the program restricted p values to four decimals, which in some cases did not go far enough.

Technical stuff only those adept at programming will ‘get’ Automatically generate DTD* files: Now these will always be up to date.

*(Document Type Definition)

New command to export runs of a specific type: Particularly useful for verification points.

Good news for network administrators New more flexible and easier-to-use license manager with greater power to serve

enterprise users: For example, network ‘seats’ can be checked out to individual laptops and multiple opening of the program on a specific computer will only use one seat.

New Designs and Name Changes in V9 of Design-Expert® Software

There have been many improvements in Design-Expert (DX9) version 9, notably split-plot designs, which

accommodate hard-to-change factors by restricting their randomization into groups. These new designs

can be seen at the bottom of the factorial design builder in DX9 (see the box in the screenshot, below

right). Randomized designs remain available, but some feature new, more descriptive names, and they

have been resorted for easier access. As always in Stat-Ease software, the most commonly used designs

get top priority, that is, they are listed in order of usefulness.

For a quick overview of the changes, compare the screenshots below.

Design-Expert V8 Software – Old Design-Expert V9 Software – New!

New Design

New Designs

Z:\Manual\DX9\DX9-02-1-Simple-Sample-FT.docx5/2/2014 9:18:00 AM

Design-Expert® software version 9: Simple Sample Design

Version 9 of Design-Expert (DX9) features a “simple sample design” that facilitates a straight-forward analysis of raw data, making it easy to calculate the mean and other statistics that characterize measurements.

To illustrate the simple sample tools of DX9, let’s characterize the performance of a motor-shaft supplier. The data, shown below, is a measure of the endplay: 61, 61, 57, 56, 60, 52, 62, 59, 62, 67, 55, 56, 52, 60, 59, 59, 60, 59, 49, 42, 55, 67, 53, 66, 60. The purchaser needs the mean, standard deviation, and 95% confidence interval of this vital attribute.

Off the Factorial tab select Simple Sample and, as shown in the screen shot, enter the Response Name “Endplay” and Rows 25 for the number of observations. Then click Continue. The program then presents a blank data entry sheet—a “design layout.”

Either type in the data now or open the file “Simple Sample-Motor Shaft.dxpx” that has it pre-entered. Now proceed with the analysis by going to the R1: Endplay node and pressing forward to the ANOVA tab. Design-Expert then presents the needed statistics as seen here.

Check out the graphs under Diagnostics (run 20 bears watching as you will see by changing to the Externally Studentized scale for the Resid vs Run chart) as well as the 95%-confidence-banded model graph copied out to the right. Also, take a look at the tool under the Post Analysis node for Point Prediction, in particular the tolerance interval, a very useful statistic for a purchaser who needs to establish incoming specifications.

This concludes our feature tour of simple sample tools in DX9. Feel free to explore other tools. If you need more information at any time, press for Tips, Screen Tips off the main menu or push the light bulb icon.

Design-Expert® SoftwareEndplay

Design Points95% CI Bands

Std # 20 Run # 20Y = Endplay = 42CI = (55.6418, 60.2782)

Run Number

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

En

dp

lay

40

45

50

55

60

65

70

Simple Sample

DX9-02-2-Gen1Factor.docx Rev. 5/2/14

Design-Expert 9 User’s Guide General One-Factor Multilevel-Categoric Tutorial 1

General Multilevel-Categoric One-Factor

Tutorial

Part 1 – The Basics

Introduction

In this tutorial you will build a general one-factor multilevel-categoric design using Design-Expert® software. This type of design is very useful for simple comparisons of categorical treatments, such as:

Who will be the best supplier,

Which type of raw material should be selected,

What happens when you change procedures for processing paperwork.

If you are in a hurry, skip the boxed bits—these are sidebars for those who want to spend more time and explore things.

Explore response surface methods: If you wish to experiment on a continuous factor, such as time, which can be

adjusted to any numerical level, consider using response surface methods (RSM) instead. This is covered in a series of

tutorials presented later in the Design-Expert User’s Guide.

The data for this example come from the Stat-Ease bowling league. Three bowlers (Pat, Mark, and Shari) are competing for the last team position. They each bowl six games in random order – ideal for proper experimentation protocol. Results are:

Game Pat Mark Shari

1 160 165 166

2 150 180 158

3 140 170 145

4 167 185 161

5 157 195 151

6 148 175 156

Mean 153.7 178.3 156.2

Bowling scores

Being a good experimenter, the team captain knows better than to simply pick the bowler with the highest mean score. The captain needs to know if the average scores are significantly different, given the variability in individual games. Maybe it’s a fluke that Mark’s score is highest.

This one-factor case study provides a good introduction to the power of simple comparative design of experiments (DOE). It exercises many handy features found in Design-Expert software.

Explore other resources: We won’t explain all features displayed in this current exercise because most will be

covered in later tutorials. Many other features and outputs are detailed only in the help system, which you can access by

clicking Help in the main menu, or in most places via a right click, or by pressing the F1 key (context sensitive).

2 General Multilevel-Categoric One-Factor Tutorial Design-Expert 9 User’s Guide

Design the Experiment

We will assume that you are familiar with your computer’s graphical user interface and your mouse. Start the program by double clicking the Design-Expert icon. You will then see the main menu and icon bar.

Click on File in the main menu. Unavailable items are dimmed. (If you prefer using your keyboard, press the Alt key and underlined letter simultaneously, in this case Alt F.)

File menu

Select the New Design item with your mouse.

Explore optional ways to select a new design: The blank-sheet icon on the left of the toolbar is a quicker path to

this screen. To try this, press Cancel to re-activate the tool bar.

Opening a new design with the blank sheet icon

Using either path, you now see four yellow tabs on the left of your screen. The Factorial tab comes up by default. Select Multilevel Categoric for this design. (If your factor is numerical, such as temperature, then you would use the One Factor option under the Response Surface tab.)

Explore what the program tells you in its annotations: Note the helpful description: “Design, also known as

“General Factorial”, for 1 to 12 factors where each factor may have a different number of levels.”

P.S. If any of your factors are quite hard to control, that is, not easily run at random levels, then consider using the Split-

Plot Multilevel Categoric design. However, restricting randomization creates big repercussion on the power of your

experiment, so do your best to allow all factors to vary run-by-run as chance dictates. (Design-Expert by default will lay

out your design in a randomized run order.)



Multilevel Categoric design

Enter the Design Parameters

Leave the number of factors at its default level of 1 but click the entry format Vertical (easier than Horizontal for multiple levels). Enter Bowler as the name of the factor. Tab down to the Units field and enter Person. Next tab to Type. Leaving Type at its default of Nominal, tab down to the Levels field and enter 3. Now tab to L(1) (level one) and enter Pat. Type Mark, and Shari for the other two levels (L2 and L3).

Multilevel Categoric design-builder dialog box – completed

Explore screen tips: For details on the options for factor type, click the light bulb icon ( ) in the toolbar to access our

context-sensitive screen tips.

Screen tips on factor Type

Press Continue to specify the remaining design options. In the Replicates field, which becomes active by default, type 6 (each bowler rolls six games). Tab to the “Assign one block per replicate” field but leave it unchecked. Design-Expert now recalculates the number of runs for this experiment: 18.


Design options entered

Press Continue. Let’s do the easy things first. Leave the number of Responses at the default of 1. Now click on the Name box and enter Score. Tab to the Units field and enter Pins.

Response name dialog box – completed

At this stage you can skip the remainder of the fields and continue on. However, it is good to gain an assessment of the power of your planned experiment. In this case, as shown in the fields below, enter the value 20 because the bowling captain does not care if averages differ by fewer than 20 pins. Then enter the value 10 for standard deviation (derived from league records as the variability of a typical bowler). Design-Expert then compute a signal-to-noise ratio of 2 (10 divided by 5).

Optional power calculator – necessary inputs entered

Press Continue to view the happy outcome – power that exceeds 80 percent probability of seeing the desired difference.

Results of power calculation

Click on Finish for Design-Expert to create the design and take you to the design layout window.



Explore the program interface: Before moving on, take a look at the unique branching interface provided by Design-

Expert for the design and analysis of experiments and resulting optimization.

Design-Expert software’s easy-to-use branching interface

You will explore some branches in this series of tutorials and others if you progress to more advanced features, such as

response surface methods for process optimization.

Save the Design

When you complete the design setup, save it to a file by selecting File, Save As. Type in the name of your choice (for this tutorial, we suggest Bowling) for your data file, which is saved as a *.dxpx type.

Save As dialog box

Click on Save. Now you’re protected in case of a system crash.

Create a Data Entry Form

In the floating Design Tool click Run Sheet (or go to the View menu and select Run Sheet) to produce a recipe sheet for your experiment with your runs in randomized order. A printout provides space to write down the responses. (Note: this view of the data does not


allow response entry. To type results into the program you must switch back to the home base – the Design Layout view.)

Run Sheet view (your run order may differ)

Explore printing features: It’s not necessary for this tutorial, but if you have a printer connected, you can select File, Print, and OK (or click the printer icon) to make a hard copy. (You can do the same from the basic design layout if you

like that format better.)

Enter the Response Data

When performing your own experiments, you will need to go out and collect the data. Simulate this by clicking File, Exit. Click on Yes if you are prompted to Save. Now re-start Design-Expert and use File, Open Design or click the open file icon on the toolbar)) to open your data file (Bowling.dxpx). You should now see your data tabulated in the randomized layout. For this example, you must enter your data in the proper order to match the correct bowlers. To do this, right-click the Factor 1 (A: Bowler) column header and choose Sort Ascending.

Sort runs by standard (std) order

Now enter the responses from the table on page one, or use the following screen. Except for run order, your design layout window must look like that shown below.



Design Layout in standard order with response data entered

When you conduct your own experiment, be sure to do the runs and enter the response(s) in randomized order. Standard order should only be used as a convenience for entering pre-existing design data.

Explore advantages of being accurate on the actual run order: If you are a real stickler, replace (type over) your run

numbers with the ones shown above, thus preserving the actual bowlers’ game sequence. Bowling six games is taxing

but manageable for any serious bowler. However, short and random breaks while bowling six games protects against

time-related effects such as learning curve (getting better as you go) and/or fatigue (tiring over time).

Save your data by selecting File, Save from the menu (or via the save icon on the toolbar). Now you’re backed up in case you mess up your data. This backup is good because now we’ll demonstrate many beneficial procedures Design-Expert features in its design layout.

For example, right click the Select button. This allows you to control what Design-Expert displays. For this exercise, choose Comments.


Select button for choosing what you wish to display in the design layout

In the comments column above we added a notation that after run 8, the bowling alley proprietor re-oiled the lane – for what that was worth. Seeing Pat’s scores, the effect evidently was negligible. ; )

Explore entering comments: Try this if you like. If comments exceed allotted space, move the cursor to the right

border of the column header until it turns into a double-headed arrow (shown below). Then, just double-click for

automatic column re-sizing.

Adjusting column size

Now, to better grasp the bowling results, order them from low-to-high as shown below by right-clicking the Response column header and selecting Sort Ascending.

Sorting a response column (also works in the factor column)

You’ll find sorting a very useful feature. It works on factors as well as responses. In this example, you quickly see that Mark bowled almost all the highest games.



Analyze the Results

Now we’ll begin data analysis. Under the Analysis branch of the program (on the left side of your screen), click the Score node. Transform options appear in the main window of Design-Expert on a progressive tool bar. You’ll click these buttons from left to right and perform the complete analysis. It’s a very easy process. The Transform screen gives you the opportunity to select a transformation for the response. This may improve the analysis’ statistical properties.

Transformation button – the starting point for the statistical analysis

Explore details on transformations: If you need some background on transformations, first try Tips. For complete

details, go to the Help command on the main menu. Click the Search tab and enter “transformations.”

As shown at the bottom of the Transform screen above, the program provides data-sensitive advice, so press ahead with the default of None by clicking the Effects tab.

Examine the Analysis

By necessity, the tutorial now turns a bit statistical. If this becomes intimidating, we recommend you attend a basic class on regression, or better yet, a DOE workshop such as Stat-Ease’s computer-intensive Experiment Design Made Easy.

Design-Expert now pops up a very specialized plot that highlights factor A—the bowlers—as an emergent effect relative to the statistical error, that is, normal variation, shown by the line of green triangles.


Initial view of the effect of Bowler

That is good! It supports what was obvious from the raw results—who bowls does matter.

Explore half-normal plots: If you want to learn more about half-normal plots of effects, work through the Two-Level

Factorial Tutorial.

To get the statistical details, press the ANOVA (Analysis of Variance) tab. Notice to the far right side of your screen that Design-Expert verifies that the results are significant.

ANOVA results (annotated), with context-sensitive Help enabled via right-click menu

Explore the ANOVA report: Now select View, Annotated ANOVA from the menu atop the screen and uncheck ()

this option. Note that the blue textual hints and explanations disappear so you can make a clean printout for statistically

savvy clients. Re-select View, Annotated ANOVA to ‘toggle’ back all the helpful hints. Before moving on, try the first

hint shown in blue: “Use your mouse to right click on individual cells for definitions.” For example, perform this tip on

the p-value of 0.0006 as shown above (select Help at the bottom of the pop-up menu). There’s a wealth of information

to be brought up from within the program with a few simple keystrokes: Take advantage!



Now click the ‘floating’ (moveable) R-squared Bookmark button (or press the scroll-down arrow at the bottom right screen) to see various summary statistics.

Summary statistics

Explore the post-ANOVA statistics: The annotations reveal the gist of what you need to know, but don’t be shy about

clicking on a value and getting online Help via a right-click (or try the F1 key). In most cases you will access helpful

advice about the particular statistic.

Now click the Coefficients Bookmark button to view the output illustrated below.

Coefficient estimates

Here you see statistical details such as coefficient estimates for each model terms and their confidence intervals (“CI”). The intercept in this simple one-factor comparative experiment is simply the overall mean score of the three bowlers. You may wonder why only two terms, A1 and A2, are provided for a predictive model on three bowlers. It turns out that the last model term, A3, is superfluous because it can be inferred once you know the mean plus the averages of the other two bowlers.

Now let’s move on to the next section within this screen: “Treatment Means.”

Treatment means

Here are the averages for each of the three bowlers. As you can see below, these are compared via pair-wise t-tests in the following part of the ANOVA report.


Treatment means

You can conclude from the treatment comparisons that:

Pat differs significantly (24.67 pins worse!) when compared with Mark (1 vs 2)

The 2.5 pins mean difference between Pat and Shari (1 vs 3) is not significant (nor is it considered important by the bowling team’s captain – recall in the design specification for power that a 10-pin difference was the minimum of interest)

Mark differs significantly (22.17 pins better!) when compared with Shari (2 vs 3).

Explore the Top feature: Before moving ahead, press Top on the floating Bookmark. This is a very handy way of

moving through long reports, so it’s worth getting in the habit of using it.

Back to the top

Analyze Residuals

Click the Diagnostics tab to bring up the normal plot of residuals. Ideally this will be a straight line, indicating no outlying abnormalities.

Explore the ‘pencil test’: If you have a pencil handy (or anything straight), hold it up to the graph. Does it loosely

cover up all the points? The answer is “Yes” in this example – it passes the “pencil test” for normality. You can

reposition the thin red line by dragging it (place the mouse pointer on the line, hold down the left button, and move the

mouse) or its “pivot point” (the round circle in the middle). However, we don’t recommend you bother doing this – the

program generally places the line in the ideal location automatically. If you need to re-set the line, simply double-click

your left mouse button over the graph.

Notice that the points are coded by color to the level of response they represent – going from cool blue for lowest values to hot red for the highest. In this example, the red point is Mark’s outstanding 195 game. Pat and Shari think Mark’s 195 game should be thrown out because it’s too high. Is this fair? Click this point so it will be highlighted on this and all the other residual graphs available via the Diagnostics Tool (the ‘floating’ palette on your screen).



Normal probability plot of residuals (195 game highlighted)

Explore the Top feature: Notice on the Diagnostics Tool that they are “studentized” by default. This converts raw

residuals, reported in original units (‘pins’ of bowling in this example), to dimensionless numbers based on standard

deviations, which come out in plus or minus scale. More details on studentization reside in Help. Raw residuals can be

displayed by choosing it off the down-list on the Diagnostics Tool shown below. Check it out!

Other ways to display residuals

In any case, when runs have greater leverage (another statistical term to look up in Help), only the Studentized form of

residuals produces valid diagnostic graphs. For example, if Pat and Shari succeed in getting Mark’s high game thrown

out (don’t worry – they won’t!), then each of Mark’s remaining five games will exhibit a leverage of 0.2 (1/5) versus

0.167 (1/6) for each of the others’ six games. Due to potential imbalances of this sort, we advise that you always leave

the Studentized feature checked (as done by default). So if you are on Residuals now, go back to the original choice that

came up by default (externally* studentized).

*P.S. Another aspect of how Design-Expert displays residuals by default is them being done “externally”. This is

explored in the Two-Level Factorial Tutorial. For now, suffice it to say that the program chooses this form of residual to

provide greater sensitivity to statistical outliers. This makes it even more compelling not to throw out Mark’s high game.

On the Diagnostics Tool, select Resid. vs. Pred. to generate a plot of residuals for each individual game versus what is predicted by the response model.

Explore an apocryphal story: Supposedly, “residuals” were originally termed “error” by statisticians, but the

management people got upset at so many mistakes being made!

Let’s make it easier to see which residual goes with which bowler by pressing the down-list arrow for the Color by option in the Diagnostics Tool and selecting A:Bowler.


Residuals versus predicted values, colored by bowler

The size of the studentized residual should be independent of its predicted value. In other words, the vertical spread of the studentized residuals should be approximately the same for each bowler. In this case the plot looks OK. Don’t be alarmed that Mark’s games stand out as a whole. The spread from bottom-to-top is not out of line with his competitors, despite their protestations about the highest score (still highlighted).

Bring up the next graph on the Diagnostics Tool list – Resid. vs Run (residuals versus run number). (Note: your graph may differ due to randomization.)

Residuals versus run chart (Note: your graph may differ due to randomization)

Here you might see trends due to changing alley conditions (the lane re-oiling, for example), bowler fatigue, or other time-related lurking variables.

Explore repercussion of possible trends: In this example things look relatively normal. However, even if you see a

pronounced upward, downward, or shift change, it will probably not bias the outcome because the runs are completely

randomized. To ensure against your experiment being sabotaged by uncontrolled variables, always randomize!

More importantly in this case, all points fall within the limits (calculated at the 95 percent confidence level). In other words, Mark’s high game does not exhibit anything more than common-cause variability, so it should not be disqualified.

Design-Expert® SoftwareScore

Color points by level ofBowler:

PatMarkShari

Std # 11 Run # 14X: 14Y: 2.175

Run Number

Ex

tern

all

y S

tud

en

tiz

ed

Re

sid

ua

ls Residuals vs. Run

-4.00

-2.00

0.00

2.00

4.00

1 3 5 7 9 11 13 15 17



View the Means and Data Plot

Select the Model Graphs tab from the progressive tool bar to display a plot containing all the response data and the average value at each level of the treatment (factor). This plot gives an excellent overview of the data and the effect of the factor levels on the mean and spread of the response. Note how conveniently Design-Expert scaled the Y axis from 140 to 200 pins in increments of 10.

One-factor effects graph with Mark’s predicted score (mean) highlighted

The squares in this effects plot represent predicted responses for each factor level (bowler). Click the square representing Mark’s mean score as shown above. Notice that Design-Expert displays the prediction for this treatment level (reverting to DOE jargon) on the legend at the left of the graph. Vertical ‘I-beam-shaped’ bars represent the 95% least significant difference (LSD) intervals for each treatment. Mark’s LSD bars don’t overlap horizontally with Pat’s or Shari’s, so with at least 95% confidence, Mark’s mean is significantly higher than the means of the other two bowlers.

Explore other points on the model graph: Oh, by the way, maybe you noticed that the numerical value for the height

of the LSD bar appeared when you clicked Mark’s square. You can also click on any round point to see the actual

scores. Check it out!

Pat and Shari’s LSD bars overlap horizontally, so we can’t say which of them bowls better. It seems they must spend a year in a minor bowling league and see if a year’s worth of games reveals a significant difference in ability. Meanwhile, Mark will be trying to live up to the high average he exhibited in the tryouts and thus justify being chosen for the Stat-Ease bowling team.

That’s it for now. Save your results by going to File, Save. You can now Exit Design-Expert if you like, or keep it open and go on to the next tutorial – part two for general one-factor design and analysis. It delves into advanced features via further adventures in bowling.


General One-Factor Tutorial

(Part 2 – Advanced Features)

Digging Deeper Into Diagnostics

(Caution: Only the more daring new users should press ahead from here—those who like to turn over every rock to see what’s underneath, that is—the types who are curious to know everything there is to know. If that’s not you, skip the rest and go on to another tutorial if it offers feature you need for your particular experiment.)

If your bowling data is active in Design-Expert® software from Part 1 of this tutorial, continue on. If you exited the program, re-start it and use File, Open Design to open data file (Bowling.dxpx). Otherwise, set up this data file as instructed above in our General One-Factor Tutorial (Part 1 – The Basics). Then, under the Analysis branch (you may already be here) click the Score node and press the Diagnostics tab.

We’re now going to look at a new graph in the Diagnostics Tool. Click the Influence option on the Diagnostics Tool palette. Then click on DFFITS. This statistic, which stands for difference in fits, measures the change in each predicted value that occurs when that response is deleted. The larger the absolute value of DFFITS, the more it influences the fitted model. (For more details on this statistic and the related deletion diagnostic, DFBETAS, see our program Help or refer to Raymond Myers’ Classical and Modern Regression with Applications, 2nd Edition (PWS Pub. Co., 1990).)

DFFITS graph (your graph may differ due to random runs)

Notice that one point lies above the rest. (The pattern on your graph may differ from what we show here due to randomized run order, but this isn’t a concern in this discussion.) The top-most point is Mark’s high game, which earlier created controversy, particularly among competitors Pat and Shari. Mark’s point falls far below a relatively conservative high benchmark of plus-or-minus two for the DFFITS. So, taking all other diagnostics into consideration, we don’t advise that this particular run be investigated further.



Nevertheless, for purposes of learning how to use new Design-Expert software features, right-click Mark’s top point with your mouse and select Highlight Point as shown below.

Highlighting a point

Myers demonstrates mathematically that the DFFITS statistic is really the externally studentized residual multiplied by high leverage points. Click the Leverage button and you’ll see that all runs exhibit equal leverage here because an equal number of runs were made at each treatment level (all three bowlers rolled six games each).

Leverages

Therefore, this DFFITS exhibits a pattern identical to that shown on the externally studentized residual graph, which you studied in the preceding tutorial. The reason we’re reviewing this is to set the stage for what you’ll do later in this tutorial – unbalance the leverages to make this session more significant for diagnostic purposes.

Explore the Pop-Out View feature: Now is a good time to go back to the DFFITS plot and press the Pop-Out View

button the very bottom of the Diagnostics Tool.

Pop-Out View button

Next go back via the Diagnostics button to the Resid. vs Run plot and verify the statement above that in this case these

two plots (DFFITs and residuals versus run) exhibit the same pattern. (You may need to press Alt-Tab to get the

windows you want on the same screen.)


Demonstration of pop-out view to see two plots side by side

You’d best now close out the pop-out view by pressing the X at the upper right corner. Otherwise your screen will get

too messy.

Here’s one final Design-Expert software feature for you before we leave the Diagnostics

Tool: Click the Report button to get a table of statistics case-by-case in standard order for the entire experiment. For those of you who prefer numbers over pictures (statisticians for sure!), this should satisfy your appetite. Notice that Mark’s high 195 game is highlighted in blue text as shown below.

Report with case statistics used for preceding diagnostics graphs

Remember, you can right-click any value in reports of this nature within Design-Expert software to view context-sensitive Help with statistical details.

Modifying the Design Layout

Design-Expert offers great flexibility when modifying data in its design layout. We’ll see in this bowling scenario how our software allows you to modify an existing design with added blocks and factor levels.

The outcome of the bowling match appears to be definitive, especially from Mark’s perspective. But Pat and Shari demand one more chance to prove themselves worthy of the



team. They still think Mark’s high 195 game was a fluke, even though this isn’t supported by the diagnostic analysis. Mark objects and a dispute ensues.

Attempting compromise, the team captain decides to toss out the highest and lowest games for each of the three bowlers and replace them with two new scores each. But Ben, a newly hired programmer and avid bowler, arrives at the alley and is allowed to participate in this second block of runs. (Yes, this makes little sense, but it will add some interest to this tour of Design-Expert’s flexibility for design and analysis of experiments – no matter how convoluted they become in actuality.)

It quickly becomes apparent that this new kid does things differently. He’s a lefty with a huge hook that’s hard to control. To aggravate this variability, Ben does something very different from other bowlers – he does not put his thumb in the ball’s hole made for that purpose. When Ben’s odd approach works, the pins go flying. But as likely as not, that ball slides off into the left gutter or careens over the edge on the right.

The results for Ben and the three original bowling team candidates are below.

Block Game Pat Mark Shari Ben

1 1 160 165 166 NA

1 2 150 180 158 NA

1 3 140 170 145 NA

1 4 167 185 161 NA

1 5 157 195 151 NA

1 6 148 175 156 NA

2 1 162 175 163 200

2 2 153 180 166 130

Bowling scores with high and low games replaced by two new games (plus a new guy)

To enter this new data (and ignore some of the old), click the Design node near the upper left of your screen. You should now see the bowling data from the first tutorial. Mark’s high 195 game remains highlighted in blue text (assuming you clicked on it as instructed on page 2 of this tutorial while performing the diagnostics).

Right click the Select column header and click Block. This design attribute is now needed to accommodate the new bowler’s (Ben’s) incoming score data.


Selecting block to display it as a column in the design layout

Right click the Response column header and choose Sort Ascending. You did this before in Part 1 so you now have this feature mastered…we hope. ; )

Mark’s best game now drops to the very bottom. Let’s single him out first to placate Pat and Shari. Right-click the square button at the left of the last row (Mark’s 195 score). Click Set Row Status, then Ignore as shown below.

Ignoring Mark’s high game

By the way, it’s OK to change your mind when modifying your design layout: You can ‘un-ignore’ a row by clicking Set Row Status, Normal.

Now let’s really get Pat’s and Shari’s hopes high by excluding their low games from consideration. Click the square button (in the Select column) to the left of the top row (Pat’s low 140 game) and, while pressing down the Shift key, also click the button in the Select column’s second row (Shari’s low 145 game). Release the Shift key. Keep your mouse within the Select column’s first or second row, right-click and choose Set Row Status, Ignore for these two low games, as shown below.



Ignoring the low games for Pat and Shari

Now move down a few rows and click the square button in the Select column’s row showing Mark’s low 165 game.

Notice the two rows below Mark’s low 165 game – the high games for Shari (166) and Pat (167). It’s now time for Shari and Pat to pay the price for complaining. While first pressing and holding down the Shift key, click the following two square buttons in the Select column’s row: Shari’s high 166 game and Pat’s high 167 game. Release the Shift key. Three rows should now be highlighted in light blue as shown below. Keep your mouse within the Select column’s highlighted three rows, right-click and choose Set Row Status, Ignore.

Ignoring Mark’s low game and the high games for Shari and Pat

Now let’s restore the original layout order. Right-click the Factor 1 (A: Bowler) column header, then choose Sort Ascending. Compare your screen with what we show below. If there are differences, fix them now to match this screenshot. However, remember that the run number is random, so you don’t need to fix that.


Back to standard order after low and high games ignored for each bowler

Now create a new block (needed for the second round of bowling) by right-clicking the Block column header and choosing Edit Info as shown below.

Creating a new block

You’ll see a form allowing you to assign names to the block(s). Don’t bother doing this now. As shown below, change Number of Blocks at the top to 2. Press the Tab key to see the change take effect. (If the name field truncates, click and move the right border of the column header to re-size it.)



Adding a second block of runs

Click OK. It seems that nothing changed, but actually the program now knows that you will be conducting another block of runs.

Now you are ready to begin adding and/or duplicating rows. This can be accomplished in different ways, depending on your ingenuity. We’ll follow routes revealing as many of the editing features as possible, although they may not demonstrate the most elegant approaches. As shown below, right click the Select column’s square button at the left of the first row (Pat’s 160 game) to bring up the editing menu. Click the first selection, Insert

Row.

Inserting a new row

You now see a new row containing blanks for the bowler and the score. (Don’t worry if it’s being ignored – crossed out, that is – for the moment.) Click the first row’s block cell directly below the block field header, then click the list arrow. Select Block 2 as shown below.

Changing the block number

Click the blank field for bowler and press the list arrow (). Select Pat. (We’re using categorical factors here, but if this were a numerical field, you’d enter a value.)


Entering a categorical value for factor

Again, right-click the Select column’s square button at the left of the first row to bring up the editing menu as shown below. Click Duplicate.

Duplicating a row

Design-Expert may pop up a warning like the one shown below.

Warning about categoric contrasts

The program is recognizing a potential problem here and is alerting you that only one bowler is in the second block. You need not worry at this stage because you will be adding others. Click the check option Do not show this warning again. This will save you aggravation later. Don’t worry – you will not be unprotected indefinitely. This warning will be re-enabled the next time you start the program.

Turning off a warning (it will come back the next time you run the program)

Press OK to proceed.

Right-click the Block column header and choose Sort Ascending.

Two new rows are now seen at the bottom of your design layout. We need two new rows apiece for Shari and Mark. Let’s simply duplicate Pat’s two new rows and update the names. Do this by first clicking the Select column’s square button at the left of Pat’s first new row, so it is highlighted. Then while holding down the Shift key, click the Select column’s square button at the left of Pat’s second new row. Both rows should now be highlighted. (This is a bit tricky, but it saves time.)



Now right-click any Select column’s square button at the left of the highlighted block and select Duplicate. (If the warning screen pops up again, click OK.)

Duplicating a block of rows

In the first duplicated row, click the field for Bowler and select Mark.

Changing name of bowler

Do the same for the last row. You now should have two new rows for both Pat and Mark. Click the Select column’s square button at the left of Mark’s first new row, so it is highlighted. Then while holding down the Shift key, click the Select column’s square button at the left of Mark’s second new row. Both rows should now be highlighted. As before, right-click any Select column’s square button at the left of the highlighted block and select Duplicate.

Duplicating two more rows

In the first duplicated row, click the field for Bowler and select Shari. Do the same for the last row.


Completing lineup for block 2 – the second round of bowling

But what about the new kid – Ben? We need to identify him as a new competitor in this bowling contest. Do this by right-clicking the header for Bowler and selecting Edit Info.

Getting ready to add a new level for the factor

Change Number of Levels to 4 (see below left).

Adding another bowler

Press Tab once. Click the field intersecting at Name column and row 4 (below right). Type the name Ben.

Entering the new bowler

Press OK. Now duplicate two more rows by clicking the Select column’s square button at the left of the first of Shari’s two new games at the bottom of the list. While holding down the Shift key, click the Select column’s square button at the left of the last run. Finally, right-click any Select column’s square button at the left of the highlighted block and select Duplicate.

In both of these new duplicated rows, click the fields for Bowler and select Ben.



Ben now on the list as a bowler

An important aside: Always randomize your run orders for actual experiments. For purposes of this tutorial, this will just be a bother, so do this only if you wish to try it out, but it’s very easy to do – simply right-click the Run column-header and do this for Block 2 as shown.

How to randomize the run order in the second block

To make it easier to enter the results, right-click the Factor 1 (A: Bowler) column header and Sort Ascending. Then right-click the Block column header and Sort Ascending. Now enter the eight new scores as shown below.

Data entered for second block of games

Go ahead now and re-analyze your data by clicking the Score node under Analysis. Move through Transform and click on the Effects tab. A warning pops up that the design is not “orthogonal.”


Warning about design now being non-orthogonal

This is a mathematical artifact of our ad hoc addition of runs in a second block. It will not create any material impact on the outcome so just press on via OK and click the square appearing at the end of the green triangles (error estimates) on the half-normal plot of effects. This puts A-Bowler in your model.

Bowler picked on half-normal plot of effects

Proceed to ANOVA (overlooking this model being not significant) and then to Diagnostics. As you will see, something is abnormal about this data. Do you notice that the residuals now line up very poorly, especially at the extreme points as shown below? On the floating Diagnostics Tool change Color by to A: Bowler.



Diagnostics for bowling results – part two: Normal plot with poorly aligned residuals

Now (referring to the color key at the left of the plot) you see that the results from Ben do not fit with the others (his games are the two outliers – low and high). Considering his odd, unstable style of bowling, this should be no surprise. Click the Resid. vs Run button to bring up the externally studentized residuals – a good tool for detecting outliers. Drag your mouse over Ben’s residuals at the far right.

Ben’s games highlighted for being outliers

Both points should now be highlighted. We must ignore or delete them. (Sorry Ben, odd behavior by programmers is considered normal at Stat-Ease, but not when it comes to bowling!)

Click the Design node (upper left) to get back to the home base of the design layout. Notice that Ben’s games are conveniently highlighted in blue text so they can easily be deleted.

Explore an option for ignoring data: It provides no advantage in this case, which features only one response

measure, but you can ignore a specific result by right-clicking that cell and setting Set Cell Status to Ignore as shown

below.


Ignoring a single cell – an option that’s not recommended for this case

In this case you could ignore his entire runs (we explained how to do this earlier). Better yet, simply delete them altogether. No offense to Ben, but given that he only bowled two games and his unorthodox style creates such abnormal variability, it is best now to click the Select column’s square button at the left of his first score of 200 (making him feel really bad ), shift-click the button below it for the second game of 130 (not so sorry to see this gone!), then without moving your mouse, right-click and select Delete Row(s).

Deleting Ben’s games

Click Yes on the warning that pops up about deleting rows (a safety precaution) and OK when asked to re-sequence the runs to fill the gap. Then go ahead and re-analyze the results.

It turns out that the added games cause no change in the overall conclusions as to who’s the better bowler. Mark remains on top. It would now be appropriate to recover the low and high games for each bowler from block 1. Because this data was not deleted, only ignored, getting it back is simply a matter of right-clicking to the left of each of the six suspect rows and changing Set Row Status to Normal. (Or, if you’re adept at manipulating lines of text or data with your mouse, do all rows at once using a click and shift-click.) Give this a try! Then re-analyze one last time.

By working through this exercise, you now see how easy it is to manipulate Design-Expert’s design layout.

P.S. Still feeling bad about deleting Ben’s scores? Don’t worry – he gets to bowl with Pat and Shari in a lesser league. After bowling for an entire year (roughly 100 games), it will become clear whether Ben’s crazy way of bowling will pay off by achieving a good average overall. After all, his 2 game average of 165 wasn’t so bad, just inconsistent (high variability). With more data, his true ability will become more apparent.


Design-Expert 9 User’s Guide General Multilevel-Categoric Factorial Tutorial 1

General Multilevel-Categoric Factorial Tutorial

Part 1 – Categoric Treatment

Introduction – A Case Study on Battery Life

Design-Expert® software version 9 offers a “Multilevel Categoric” option, also known as a “general factorial” on the “Factorial” design tab. If you have completed the General One-Factor Multilevel-Categoric Tutorial (recommended), you’ve seen how this option handles one multilevel, categorical factor. In this two-part tutorial you will learn how to set up a design for multiple categorical factors. Part 2 shows you how to convert truly continuous factors, such as temperature, from categorical to numerical. With this you can generate response surface graphs that provide a better perspective of your system.

If you are in a hurry, skip the boxed bits—these are sidebars for those who want to spend more time and explore things.

The experiment in this case, which comes from Montgomery’s Design and Analysis of Experiments, seeks consistently long life in a battery that will be subjected to extremes in ambient conditions. It evaluates three materials (factor A) at three levels of temperature (factor B). Four batteries are tested at each of the nine two-factor combinations in a completely randomized design. The responses from the resulting 36 runs are shown below.

Material

Type

Temperature (deg F)

15 70 125

A1 130 155 34 40 20 70

74 180 80 75 82 58

A2 150 188 136 122 25 70

159 126 106 115 58 45

A3 138 110 174 120 96 104

168 160 150 139 82 60

General factorial design on battery life (data in units of hours)

The following questions must be answered:

• How does material type and temperature affect battery life?

• Do any materials provide uniformly long life regardless of temperature?

The big payoff comes if the battery can be made more tolerant to temperature variations in the field.

This case study provides a good example of applying statistical DOE for robust product design. Let’s get started on it!


To build the design, choose File, New Design as shown below (or to save strokes, simply click the blank-sheet icon () on the toolbar).

2 General Multilevel-Categoric Factorial Tutorial Design-Expert 9 User’s Guide

Starting a new design via menu (option: click blank-sheet icon () on the toolbar)

Then from the default Factorial tab, click Multilevel Categoric. Choose 2 as the number of factors. If you are in Horizontal entry mode, change it to Vertical. (Design-Expert will remember this the next time you set up a design.)

Selecting number of factors for multilevel-categoric general-factorial design

Enter Material for factor name A (Categoric). Key in the word Type as your Units. Enter the value 3 for the number of levels. Change the treatment names to A1, A2 and A3. Notice that Type in the far left column defaults to Nominal (named) as opposed to ordinal (ordered). This difference in the nature of factors affects how Design-Expert codes the categorical levels, which changes the model coefficients reported under ANOVA in the subsequent response analysis. Your design should now appear as that shown below.

Entering material as a nominal factor

Explore details in Help on general factorial designs: Tutorials such as this one on general factorials will quickly get

you up to speed on how to use Design-Expert software, but it does not serve as a statistical primer for design and

analysis of experiments. If you crave such details, Help is at your fingertips! For example, go to Help, Contents and

work your way down the tree structure through the factorial branches to General (Multi-Level) Factorial Design. Note

the details on the distinction in categoric contrasts (Nominal vs Ordinal).



Help on general factorial design

Close Help by pressing X on its window.

Now enter factor B data by keying in Temperature for factor name B (Categoric), deg F for units, 3 for the number of levels, and 15, 70 and 125 for the levels. Press Nominal, click the arrow on the drop list, then choose Ordinal as shown below. This change from Nominal to Ordinal indicates that although this factor is being treated categorically (for example, due to controls offering only the three levels), temperature is really a continuous factor. Click Continue at the screen’s far lower right.

Entering information on factor B

Enter 4 for replicates. The number of runs (36) won’t be updated until you press the Tab key or move from the cell. Leave the blocks option alone because these experiments are completely randomized.


Entering the number of replicates

Click Continue to move on to the entry screen for responses. Leave the default responses at 1. Enter name as Life and units as hours.

Now we will walk you through a calculation of power – the ability of your experiment to detect meaningful differences in treatments. If you do too few runs and under-power your experiment, an important change in response (the “signal”) will become obscured by normal system/test variation (the “noise”). That would be a waste of time and materials. Design-Expert makes the calculation of power easy and puts it in upfront in the design-building process so you have a chance to bolster your experiment, if necessary. Let’s assume that battery life must improve by at least 50 hours to be of any interest and that quality control records produce a standard deviation of 30. Enter these values as shown below, Tab (or click) out of 30, and Design-Expert then calculates the signal to noise ratio.

Response entry screen

Press Continue to see the power of this design for the difference that the engineers hope to detect, at a minimum. It is calculated to be 94.5 % probability of seeing a difference (delta) as small as 50 hours. This exceeds the rule-of-thumb for power of 80 % at a minimum, thus it can be concluded that the planned design will suffice.

Power calculation

Click Continue to complete the design specification process. Design-Expert now displays the 36 runs (in random order) from the replicated 3x3 factorial design.

Analyze the Results

To save time, simulate the experimental results by right-clicking the response header and selecting Simulate.

Explore simulation tools: A heads-up for statistics educators: You can build your own simulations via the Design

Tools. Feel free to bring up the controls for this and press Help for details on using it.



Choosing a simulation

Then choose Load an existing simulation. Finally, click the file named Battery.simx and Open it. If this file does not appear, e-mail [email protected] for help. You should now see data from the experiment . This is a good time to preserve your work: Select File and Save As. Change the file name to Battery.dxpx and Save.

Now go to the Analysis branch of the program and click the node labeled R1:Life. This brings up options for applying response transformations.

First step in the analysis – transformation options

Leave the transformation at the default of “None” and go ahead and click the Effects button displayed next in the toolbar for response analysis. Design-Expert now provides an initial effect selection and displays it graphically on a specialized statistical plot called a “half-normal.”



Initial effect selection

Explore details on how the half-normal plot is constructed for general factorial designs: The program displays the

absolute value of all effects (plotted as squares) on the bottom axis. The procedure is detailed in a presentation by

Patrick Whitcomb on “Graphical Selection of Effects in General Factorials” (2007 Fall Technical Conference co-

sponsored by the American Society for Quality and the American Statistical Association) – contact Stat-Ease for a copy.

Design-Expert pre-selected two outstanding effects – the main effects of factors A and B. You can, and in this case should, modify the default effect selection. Move your mouse cursor over the unlabeled square and click it. (Note that this goes both ways, that is, you can deselect chosen effects with a simple mouse click.)

Another effect chosen

Interaction AB is now identified. Notice that Design-Expert adjusts the line to exclude the chosen effects. You will gain more practice on the use of half-normal plots for picking effects in the Two-Level Factorial Tutorial. It’s best to now press ahead in this case.

Explore the Effects List: For statistical details, press the Effects List bar on the floating Effects Tool.

Effects list



Notice the designation “M” for the selected model terms A, B and AB and the “e” next to the pure error line in this

statistical spreadsheet. You may be wondering why there are so many estimates of pure error. (If not, skip ahead!)

Each subgroup of 4 replications provides 3 degrees of freedom (“df”) of pure error. This was done for all 9 factor

combinations (3x3) which yields 27 df (= 3*9) in total for estimating pure error.

This screen provides many features for model selection, which will be covered in tutorials on response surface methods

(RSM).

Click the ANOVA tab to see the analysis of variance for this chosen model. If you do not see annotations in blue text as shown below, select View, Annotated ANOVA.

Annotated ANOVA Report

Explore the other details provided under the ANOVA tab: Scroll down or press bookmarks on the floating tool to see

post-ANOVA statistics such as R-Squared. As you can conclude for yourself by reading the annotations, the results look

good. Further down the report are details of the model based on nominal contrasts. We provide a breakdown on this in

the Experiment Design Made Easy workshop. To keep this tutorial moving, it’s best not to get bogged down in the

mathematics of modeling categorical factors, so press ahead

Open the Diagnostics tab and examine the residual graphs. By default you see the normal plot of residuals, which ideally fall more-or-less in line. The pattern here is a bit askew but not badly abnormal, so do not worry.


Normal plot of residuals – looks OK

Explore all the other diagnostic plots: Take some time now to click your way down the floating tool and then over to

the Influence side for viewing every one of the diagnostics plots and the final report.

The tedious, but necessary, model-fitting and statistical validation is now completed, so you are free and clear to finally assess the outcome of the experiment and decide whether any materials provide uniformly long battery-life regardless of temperature.

Present the Experimental Findings

Click the Model Graphs to view the long-awaited results. Design-Expert automatically presents the AB interaction plot – identified by the Term window on the floating Factors Tool.

Default model graph – interaction plot with A on bottom (X1) axis



Explore an over-simplified factor plot: Choose the One Factor plot view via the View menu, or by simply pressing

the appropriate button on the floating Graphs Tool. Another way to bring up a one-factor plot (the main effect of A or B,

in this case) is by clicking the down-list arrow () for the Term selection on the Factors Tool. Try all these approaches

if you like, but expect to be warned about presenting main effects of factors that interact. This can be very misleading.

In this case it will be a mistake to look at either material or temperature effects alone, because one factor depends on the

other. However, while you are at this, explore options on the Factors Tool for B: Temperature. Press the three buttons

from left to right and notice how much the effects change due to the interaction. The press avg.

Viewing each Temperature and then its average for the effect plot of Material

Notice how the least significant difference (LSD) bars contract after the averaging. But nevertheless, this is not helpful

because it obscures the interaction. On the Graphs Tool press the Interaction plot to bring back the true picture.

Graphs tool – interaction plot selected

Right click the Temperature factor on the floating Factors Tool and change it to the X1 Axis, thus producing an interaction graph with the ordinal factor displayed in a continuous manner and the nominal factor (material) laid out discretely as separate lines. This makes it easier to interpret your results.

Effect graph with temperature on bottom axis


Explore how the software identifies points: Click the highest point (green) at the upper left of the graph.

Point highlighted for identification

Note how to the left of the plot the software identifies the point by:

Standard order number (2) and run number (randomized)

Actual response “Y” (188)

Factor levels “X” (temperature of 15 with material A2).

The actual results are represented by various-colored circles. If there are multiples, the program displays a number; in

this case quite a few labeled “2”. Click these points multiple times to see details on each and every one of them. You

can also click on the non-circular symbols (square, triangle or diamond) to display the predicted outcome and least

significant difference (LSD). Try this!

To produce a cleaner looking plot, go to View and deselect Show Legend. Now let’s do some more clean-up for report purposes: Right-click over the graph and select Graph

Preferences.

Right-click menu selection for graph preferences

Now click the All Graphs tab and turn off (uncheck) the Show design points on graph option, as shown below.

Turning off design points

Press OK.

Explore copy/past to Microsoft Word (or the like): This is an optional sidetrack on this tutorial: To have your graph

look like that shown below for reporting purposes, do the following: Edit, Copy from Design-Expert, then Paste into

Microsoft Word.



Clean-looking interaction graph

From this graph you can see that all three materials work very well at the low temperature (15 degrees). Based on the overlapping LSD bars, it would be fair to say that no material stands out at this low temperature end of the scale. However, the A1 material clearly falls off at the 70 degree temperature, which would be encountered most often, so it must be rejected. None of the materials perform very well at the highest temperature (125 degrees), but the upper end of the LSD bar for A2 barely overlaps the bottom end of the LSD bar for A3. Therefore, with respect to temperature sensitivity, material A3 may be the most robust material for making batteries.

Finally, if you do have an opportunity to present graphics in color, here’s a dazzling new and easy way to display general factorial effects with Design-Expert: Click 3D Surface on the floating Graphs Tool.

Now place your mouse cursor on the graph – notice that it changes to a hand (). While pressing the left mouse button, spin the graph so the temperature axis is at the bottom. (Alternatively, to match our graph most precisely, select View, Show Rotation and enter coordinates of h (horizontal) 20 and v (vertical) 80.)

B: Temperature (deg F)

A: Material (Type)

15 70 125

Life

(h

ou

rs)

0

50

100

150

200

A1

A2

A3

Interaction


3D surface plot – rotated slightly for a better view

The 3D view presents a different perspective of the general factorial effects – more on a macro level of the overall experimental landscape. Now the inferiority of material A1 (red bars) becomes obvious: The other two materials tower over it at the mid-temperature of 70 degrees F. Clearly the next step is to eliminate material A1 from contention and perhaps do some further investigations on A2 and A3.

A1

A2

A3

15 70

125

0

50

100

150

200

Lif

e (

ho

urs

)


A: Material (Type)



General Factorial Tutorial

(Part 2 – Making Factors Numeric)

Continued – A Case Study on Battery Life

In the preceding, General Factorial Tutorial – Part 1, you treated all factors categorically. The main purpose of Part 2 of this tutorial is to illustrate some of the functions built into Design-Expert that can be used to make effects graphs better fit the type of data we’re dealing with, in addition to making the results easier to interpret and understand. (We’ve already said that Montgomery’s classic battery experiment could have been handled by using the Response Surface tab in Design-Expert software and constructing a one-factor design on temperature, with the addition of one categorical factor at three levels for the material type. Never fear, we’ll cover response surface methods like this from the ground up later in the Response Surface Tutorials.) But to demonstrate the flexibility of Design-Expert, let’s see how you can make the shift from categoric to numeric after-the-fact, even within the context of a general factorial design.

To get started, re-open the file named Battery.dxpx saved earlier. If the file is already open, then click the Design node.

Changing a Factor from Categoric to Numeric

Place your cursor on the B:Temperature factor column heading, right click, and move your cursor to Make Numeric and then Continuous Numeric. (The “discrete” option could also work in this case. It works well for numeric factors that for convenience-sake must be set at specific levels. For example, imagine that the testing chamber for the batteries has only three settings – 15, 70 and 125.)

Options for editing a factor

The software pops up a warning not to do this if the factor really is categorical. To acknowledge it and move on, click the OK button below the message.

Re-Analyzing the Results

To re-analyze your data, click the analysis node labeled Life. Then click the Model tab. When you designated a factor being numeric, Design-Expert automatically shifted to fitting a polynomial, such as those used for response surface methods. To model non-linear


response to temperature (factor B), double-click the AB2 term, or via a right click add it to the Model as shown below. (Squared terms capture curvature.)

Model selection screen

Click the ANOVA tab. You will get a warning about hierarchy.

Hierarchy warning

This warning arises because you chose a higher order term without support by parent terms, in this case: AB and B2. Click Yes and move on.

Explore statistical reasons for maintaining model hierarchy: For details, search out the topic on “Model Hierarchy

Check” in the Help System.

The ANOVA report now displays in the view (annotated or not) that you used last. By comparing this output with the ANOVA done in Part One, observe that the lines for the model and residual come out the same, but the terms involving B differ. In Part One we treated factor B (temperature) categorically, although in an ordinal manner. Now that this factor is recognized explicitly as numeric, what was the effect of B is now broken down to two model terms – B and B2, and AB becomes AB plus AB2.

ANOVA output



The whole purpose of this exercise is to make a better looking effects graph. Let’s see what this looks like by clicking the Model Graphs tab. Go to the floating Factors Tool, right-click B (Temperature) and change it to the X1 Axis.

Viewing the interaction with temperature on X-axis

You now have a plot characterizing the data from Part One of this ongoing case study, except that the above lines are now continuous with temperature, whereas in Part One they were displayed as discrete (categorical) segments. Notice that the curves by temperature (modeled by B2) depend on the type of material (A). This provides graphical verification of the significance of the AB2 term in the model.

The dotted lines are the confidence bands, which in this case cause more trouble than they’re worth for conveying the uncertainty in fitting, so let’s change Graph Preferences to take these off. First, right-click over the graph and select Graph Preferences. Now click the XY Graphs tab and in the Polynomial Graphs section click the None option as shown below.

Turning off confidence bands


Press OK. That cleans up the picture, but perhaps too much! Let’s put the points back on for perspective. Again, right-click over the graph and select Graph Preferences. Now click the All Graphs tab and turn on (check) the Show design points on graph option. Your plot should now look like that copied out below.

Presentation of battery life made of various materials versus temperature on continuous scale

The conclusions remain the same as before: Material A3 will maximize battery life with minimum variation in ambient temperature. However, by treating temperature numerically, predictions can be made at values between those tested. Of course, these findings are subject to confirmation tests.

Postscript: Demo of “Pop out” View

Before exiting Design-Expert, give this a try: Press the Pop out View on the floating Graphs Tool (or go to the View menu and select Pop out View).

Pop out View

This pushes the current graph out of its fixed Windows pane into a ‘clone’ that floats around on your screen. Now on the original Factors Tool right click on A (Material) and return it to the X1 Axis. Then do an Alt-Tab to bring back the clone of the previous view back on your current window.


A: Material (Type)

15.00 37.00 59.00 81.00 103.00 125.00

Life

(h

ou

rs)

0

50

100

150

200

22

2

2

2

A1

A2

A3

Interaction



Two ways of viewing the battery life results

You can present Design-Expert outputs both ways for your audience:

Curves for each material as a function of temperature on the X1 axis, or

Two temperature lines connected to the three discrete materials as X1.

Another way to capture alternative graphs is to copy and paste them into a word-processor, spreadsheet, or presentation program. Then you can add annotations and explanations for reporting purposes.

DX9-03-1-TwoLevel.docx Rev.5/2/14

Design-Expert 9 User’s Guide Two-Level Factorial Tutorial 1

Two-Level Factorial Tutorial

Introduction

This tutorial demonstrates the use of Design-Expert® software for two-level factorial designs. These designs will help you screen many factors to discover the vital few, and perhaps how they interact. If you are in a hurry, skip the boxed bits—these are sidebars for those who want to spend more time and explore things.

Explore fundamental features of the program: Before going any further with this tutorial, go back and do the one on

a General One-Factor experiment. Features demonstrated there will not be detailed here.

The data you will now analyze comes from Douglas Montgomery’s textbook, Design and Analysis of Experiments, published by John Wiley and Sons, New York. A wafer-board manufacturer must immediately reduce the concentration of formaldehyde used as a processing aid for a filtration operation. Otherwise they will be shut down by regulatory officials. To systematically explore their options, process engineers set up a full-factorial two-level design on the key factors, including concentration at its current level and an acceptably low one.

Factor Units Low Level (–) High Level (+)

A. Temperature deg C 24 35

B. Pressure psig 10 15

C. Concentration percent 2 4

D. Stir Rate rpm 15 30

Factors and levels for full-factorial design example

At each combination of these process settings, the experimenters recorded the filtration rate. The goal is to maximize the filtration rate and also try to find conditions that allow a reduction in the concentration of formaldehyde, Factor C. This case study exercises many of the two-level design features offered by Design-Expert. It should get you well down the road to being a power user. Let’s get going!

Explore what to do if a factor like temperature is hard to change: Ideally the run order of your experiment will be

completely randomized, which is what Design-Expert will lay out for you by default. If you really cannot accomplish

this due to one or more factors being too hard to change that quickly, choose the Split-Plot design. However, keep in

mind that you will pay a price in reduced statistical power for the factors that become restricted in randomization.

Before embarking on a split plot, take the “Feature Tour” to get an orientation on how Design-Expert designs such an

experiment and what to watch out for in the selection of effects, etc.


Start the program by finding and double-clicking the Design-Expert icon. Select File, New Design.

2 Two-Level Factorial Tutorial Design-Expert 9 User’s Guide

Explore other options for selecting a new design: Click the blank-sheet icon on the left of the toolbar, or click New

Design as shown in red below.

Start-up page – New Design option highlighted in red

You now see four tabs to the left of your screen. Stay with the Factorial choice, which comes up by default. You’ll be using the default selection: Randomized Regular Two-Factorial.

Two-level factorial design builder

Explore the design builder: Design-Expert’s design builder offers full and fractional two-level factorials for 2 to 21

factors in powers of two (4, 8, 16…) for up to 512 runs. The choices appear in color on your screen. White squares

symbolize full factorials requiring 2k runs for k (the number of factors) from 2 to 9. The other choices are colored like a

stoplight: green for go, yellow for proceed with caution, and red for stop, which represent varying degrees of resolution:

≥ V, IV, and III, respectively. For a quick overview of these color codes, press the screen tips button (or select Tips,

Screen Tips) and click topic 1: “What type of information do you want?”

Screen tips for factorial design builder

Close the Screen Help by pressing X at the upper right of the pop-up window. You now see that the notation shown on

the non-white boxes is “2k-p,” where p designates the fraction of the design. For example, here’s the anatomy of a 25-1

design:

5 factors will be tested each at two levels.

A 2-1 or one-half (1/2) fraction, with the optimal resolution, will be selected from the original 25 (32) combinations,

thus this option appears in the 16-run row (one-half of 32).

For complete details on fractional factorials, and the concept of resolution, refer to the Montgomery textbook or see

Chapter 5 in DOE Simplified, 2nd Edition (Anderson, Whitcomb, Productivity Press, NY, NY, 2007). To gain a working

knowledge of two-level designs, attend Stat-Ease’s computer-intensive workshop on Experiment Design Made Easy.



Let’s get on with the case at hand – a full-factorial design. Click the white square labeled 24 in column 4 (number of factors) in the Runs row labeled 16. It turns black once it is selected, as shown below.

Selecting a full, two-level design on four factors which produces 16 runs

Explore other elements of design: At the bottom of the screen you see options to select the number of Replicates of

the design, the number of Blocks, and the Center points per block. Leave them at their defaults of 1, 1 and 0;

respectively.

Click the Continue button. You can now enter the names, units of measure, and levels for your experimental factors. Use the arrow keys, tab key, or mouse to move from one space to the next. The Tab (or Shift Tab) key moves the cursor forward (or backward) between entry fields. Enter for each factor (A, B, C and D) the Name, Units, Low and High levels shown on the screen shot below.

Factors – after entering name, units, and levels (plus a peek at options for “Type”)

Explore how to enter alphanumeric levels: Factors can be of two distinct types – “Numeric” or “Categoric.”

Numeric data characterizes a continuous scale such as temperature or pressure. Categoric data, such as catalyst type or

automobile model, occurs in distinct levels. Design-Expert permits characters (for example, words like “Low” or

“High”) for the levels of categorical factors. You change the type of factor by clicking on cells in the Type column and

choosing “Categoric” from the drop down list, or by typing “C” (or “N” for numeric). Give this a try – back and forth!

Leave the default as “Numeric” for all factors in this case.

Now click Continue to bring up the Responses dialog box. With the list arrow you can enter up to 999 responses (more than that can be added later if you like). In this case we only need to enter a single response name (Filtration Rate) and units (gallons/hour) as shown below.


Response values entered

It is good to now assess the power of your experiment design. In this case, management does not care if averages differ by less than 10 gallons per hour (there’s no value in improvements smaller than this). Engineering records provide the standard deviation of 5 (the process variability). Enter these values as shown below. Design-Expert then computes the signal to noise ratio (10/5=2).

Power wizard – necessary inputs entered

Press Continue to view the positive outcome – power that exceeds 80 percent probability of seeing the desired difference.


Click Finish to accept these inputs and generate the design layout window.

You’ve now completed the first phase of DOE – the design. Notice that this is one of four main features (branches) offered by Design-Expert software, the others being Analysis, Optimization and Post Analysis (prediction, confirmation, etc.).



Explore how to identify your experiment: Click the Notes node to take a look at what’s written there by default. Add

your own comments if you like.

Notes on data file

Exit the notes page by clicking the Design node. (Notice the node appears as Design (Actual) – meaning your factors are

displayed in actual levels, as opposed to coded form).

You’ve put in some work at this point so it is a good time to save it. The quickest way of doing this is to press the standard save icon .

Explore another way to save your data: You can also go to the File menu and select Save As. Type in the name of

your choice (such as Factorial) for your data file. Note the default Save as type being *.dxpx — the format peculiar to

this version of Design-Expert. Then click Save.

Saving the design

Enter the Response Data

At this stage you normally would print the run sheet, perform the experiments, and record the responses. The software automatically lists the runs in randomized order, protecting against any lurking factors such as time, temperature, humidity, or the like. To avoid the tedium of typing numbers, yet preserve a real-life flavor for this exercise, simulate the data by right-clicking the Response column header to bring up a new menu. Select Simulate and then Finish to Load and existing simulation (the default).


Simulating the response

You will now see a list of simulation files. Double-click Filtrate.simx (or press Open). The filtration process simulation now generates the response data. Right click the Std column header (on the gray square labeled Std) and select Sort Ascending as shown below.

Sorting by standard (Std) order

Your data should now match the screen shot shown below except for a different random run order. (When doing your own experiments, always do them in random order. Otherwise, lurking factors that change with time will bias your results.)

Design layout in standard order – response data entered (via simulation)

Explore how to adjust column widths: Note that the Column Header for Factor 3 is truncated (ie. Concentrat…

percent). To automatically re-size the column, move the cursor to the right border of the column header until it turns

into a double-headed arrow ( ). Double-click and the column will be resized to fit the Column Header.



Now that you’ve got responses recorded, it’s another opportune time to save the updated file by clicking the Save icon .

Explore how to change number formats: The response data came in under a general format. In some cases you will

get cleaner outputs if you change this to a fixed format. Place your mouse over the Response column heading (top of

the response column), right click and select Edit Info...

Selecting Edit Info option

Click the Format arrow and choose 0.0 from the drop list. Press OK.

Changing the format

Using the same Edit Info feature, you can also change input factors’ format, names, or levels. Try this by right clicking

any other column headings.

Design-Expert provides two methods of displaying the levels of the factors in a design: Actual levels of the factors. Coded as -1 for low levels and +1 for high levels.

The default design layout is actual factor levels in run order.

Explore how to view factors in coded format: To view the design in coded values, click Display Options on the

menu bar and select Process Factors - Coded. Your screen should now look like the one shown below.

Design layout - coded factor levels (your run order may differ)

Notice that the Design node now displays “coded” in parentheses – Design (Coded). This can be helpful to see at a

glance whether anyone changed any factor levels from their design points.

Now convert the factors back to their original values by clicking on Display Options from the menu bar and selecting

Process Factors - Actual.


Pre-Analysis of Effects via Data Sorts and Simple Scatter Plots

Design-Expert provides various ways for you to get an overall sense of your data before moving on to an in-depth analysis. For example, via the same right-click menu used to Edit Info, you can sort by any column.

To see this, move your mouse to the top of column Factor 1 (A: Temperature) and right-click. Then select Sort Ascending.

Sorting the design on a factor

You will now see more clearly the impact of temperature on the response. Better yet, you can make a plot of the response versus factor A by selecting the Graph Columns node that branches from the design ‘root’ at the upper left of your screen. You should now see a scatter plot with (by default) factor A:Temperature on the X-axis and the response of Filtration Rate on the Y-axis.

Observe by looking at the graph how temperature makes a big impact on the response. This leads to the high correlation reported on the legend.

Legend for default graph columns on filtration data

Another indicator the strong connection of temperature to filtration rate is the red color in the correlation grid at the intersection of these two variables.



Correlation grid—switching graph from factor A (left) to B (right)

To see the impact of the next factor, B, click the next square to right as shown above. Notice now that pressure has little correlation with filtration rate—this relationship turns out to be insignificant.

Explore how to use the Color By feature: To be completely diligent in exploring the experimental results graphically,

start this brief sidetrack by clicking forward to factors C and D. Then go back to the scatter plot of A versus Filtration

Rate. By default the points are colored by standard order. Just for fun, click the Color By drop-down list and select

C:Concentration as shown below.

Graph columns for temperature versus filtration rate, colored by concentration

Do you see how two colors stratify at each level of temperature – but oppositely – red at the top for the rates plotted at

the left (temperature 24) versus blue coming out higher for filtration rate at the right? Consider what this may indicate

about how concentration interacts with temperature to produce an effect on filtration rate. However, let’s not get ahead

of ourselves – this is only a preliminary to more thorough analyses using much more sophisticated graphical and

statistical tools.

You may wonder why the number “2” appears besides a few points on this plot. This notation indicates the presence of multiple points at the same location.

Explore points piled up on each other: Click on one of these points more than once to identify the individual runs

(look to the left of the graph for this detailing). Coming up soon you will use powerful analysis features in

Design-Expert software to find out what’s really going on in this wafer-board production process.


Analyze the Results

To begin analyzing the design, click the Filtration Rate response node on the left side of your screen. This brings up the analytical tool bar across the top of the screen. To do the statistical analysis, simply click the tabs progressively from left to right.

The Transform tab is initially highlighted, as shown below. It displays a list of mathematical functions that you may apply to your response.

Explore transformation options: For helpful advice on when to transform your response (or not!), press the Tips

button.

Transformation options

Near the bottom of your Transform screen, Design-Expert notes that the response range is more than two-fold (“Ratio of max to min is 2.4186”). This number falls below the ratio of 3 where “…power transforms have little effect.” Therefore, you can leave the transformation at its default: None.

Choosing Effects to Model

Click the Effects tab. The program displays the absolute value of all effects (plotted as squares) on a half-normal probability plot. (Color-coding provides details whether the effects are positive or negative.)



Half-normal plot of effects – nothing selected

Note the message on your screen: “Select significant terms – see TiP.S.” You must choose which effects to include in the model. If you proceed without doing so at this point, you will get a warning message stating “You have not selected any factors for the model.” The program will allow you to proceed, but with only the mean as the model (no effects), or you can opt to be sent back to the Effects view (a much better choice!).

You can select effects by simply clicking on the square points. Start with the largest effect at the right side of the plot, as shown below.

First effect chosen

Keep selecting individual effects from right to left until the line matches up with the majority of the effects near zero. Notice that Design-Expert adjusts the line to exclude chosen effects. At the point where you should stop, this line ‘jumps up,’ leaving a noticeable gap as shown below. In this case, the gap appears between the main effect of factor C and the near-zero effects that fall on the line.

Explore program details for selecting terms: Press the handy screen tips button (light-bulb icon. Then click the

Show Select Terms Video Clip to view how to select effects. After seeing this ‘demo’, X out of the window.

Screen Tips for half-normal plot of effects


Half-normal probability plot - all big effects selected

Until you get familiar with the half-normal plot, it is best that you continue picking effects one at a time from the largest (right-most) on down.

Explore another way to select effects: After doing this many times only one at a time, you will get tired of all the

mouse clicking. Then try ‘roping’ the obviously significant effects. To see how this feature works, first go to the

floating Effects Tool and press Clear Selection as shown below.

Clearing effects

Now, as shown in the screenshot below, do a mouse drag (depress left button and pull down) to form a box around the

effects that stand off at the right from the trivial many that are lined up near zero. This is like a cowboy roping stray

cows.

Roping the effects

Notice that when you release the mouse button Design-Expert labels the roped effects.

To really see the magnitude of the chosen effects, display them on an ordered bar chart. Do this via the floating Effects Tool, Pareto. Notice the vertical axis shows the t-value of the absolute effects. This dimensionless statistic scales the effects in terms of standard deviations. In this case, it makes no difference to the appearance of the chart, but when you encounter botched factor levels, missing data, and the like, the t-value scale provides a more accurate measure of relative effects. Click the next biggest bar and notice it is identified as ABD as shown below (lower right).

Design-Expert® SoftwareFiltration Rate

Shapiro-Wilk testW-value = 0.974p-value = 0.927A: TemperatureB: PressureC: ConcentrationD: Stir Rate

Positive Effects Negative Effects

0.00 3.60 7.21 10.81 14.42 18.02 21.63

0

10

20

30

50

70

80

90

95

99

Half-Normal Plot

|Standardized Effect|H

alf

-N

orm

al

% P

ro

ba

bil

ity

A-Temperature

C-ConcentrationD-Stir Rate

AC

AD



Pareto chart of effects with ABD picked (a mistake!)

Notice that the ABD bar falls below the bottom limit, so click off the ABD bar.

Explore statistical calculations for effects: To see quantitative detail on the chosen model effects and those

remaining for estimation of error, click Effects List.

Effects list

If you see an “M” (model designated) for ABD, double-click it back to “e” (error.) For details regarding this screen,

investigate Tips. However, it may be best to move on fairly quickly so you can finish this tutorial before being

overwhelmed with too much information (TMI). You can always return to screens later (perhaps after a suitable period

of rest) and study the details found in Tips and our program Help.

ANOVA and Statistical Analysis

It is now time to look at the statistics in detail with the analysis of variance (ANOVA) table. Click the ANOVA tab to see the selected effects and their coefficients. By default, Design-Expert provides annotations in blue text.


Explore options for annotations: These can be toggled off via the View menu. Note that if turned off, it will not tell

you if your model is significant.

Annotated View

Check the probability (“p-value”) for the Model. By default, Design-Expert considers values of ≤ 0.05 to be significant.

Explore how to change what you consider to be significant: Via Edit, Preferences, Math, the significance threshold

can be changed to 0.1 or 0.01, depending on how much risk you wish to accept. Be careful what you wish!

Inspect the p-values for the model terms A, C, D, AC, and AD: All pass the 0.05 test with room to spare.

ANOVA report

Explore context-sensitive Help: Pop-up definitions for numbers on the ANOVA table can be obtained by right

clicking and choosing Help. Try this for the Mean Square Residual statistic, as shown below.

Accessing context-sensitive Help

When you finish with Help, close it by clicking the red X in the corner.

Scroll down the ANOVA output for further statistics such as R-squared and the like. Refer to the annotations and also access Help for details. Then continue on for a look at the estimates for the model coefficients and their associated statistics. Lastly, page down to the end of the report and see the predictive equations both in coded and uncoded form.



Explore a quicker way to look through the report: Use the handy floating Bookmarks tool to move around the report.

Bookmarks quickly move you to the section you want to see

Rather than belabor the numbers, let’s move on and ultimately let the effect graphs tell the story. However, first we must do some diagnostics to validate the model.

Validate the Model

Click the Diagnostics tab to generate a normal probability plot of the residuals.

Normal plot of residuals

By default, residuals are studentized – essentially a conversion to standard deviation scale. Also, they are done externally, that is, with each result taken out before calculating its residual. Statisticians refer to this approach as a “case-deletion diagnostic.” If something goes wrong in your experiment or measurement and it generates a true outlier for a given run, the discrepant value will be removed before assessing it for influencing the model fit. This improves the detection of any abnormalities.


Explore raw residuals option: For standard two-level factorial designs like this, plotting the raw residuals (in original

units of measure) will be just as effective. On the Diagnostics Tool, pull down the options as shown and choose

Residuals (raw) to satisfy yourself that this is true (the pattern will not change appreciably).

Changing the form of residuals

We advise that you return to studentized scale in the end, because as a general rule this is the most robust approach for

diagnosing residuals.

Ideally the normal plot of residuals is a straight line, indicating no abnormalities. (The data doesn’t have to match up perfectly with the line. A good rule of thumb is called the “fat pencil” test. If you can put a fat pencil over the line and cover up all the data points, the data is sufficiently normal.) In this case the plot looks OK so move on.

From the floating Diagnostics Tool select Resid. vs. Pred. (residuals versus predicted), shown below.

Residuals versus predicted response values

The size of the residual should be independent of its predicted value. In other words, the vertical spread of the studentized residuals should be approximately the same across all levels of the predicted values. In this case the plot looks OK.



Next select Resid. vs. Pred to see a very useful plot that’s often referred to as “Outlier t” because it shows how many standard deviations (t-values) a given run falls off relative to what one would expect from all the others.

Residuals versus run plot (yours may differ due to random run order)

Design-Expert provides upper and lower red lines that are similar to 95% confidence control limits on a run chart. In this case none of the points stands out. Because this graph is plotted in randomized run order, the ordering of the points on your screen will be different than shown here. Normally, though, you should look for patterns, not just outliers. An obvious example would be a steady decrease in residuals from start to finish, in other words, a downward trend. That would be cause for concern about the stability of your system and merit investigation. However, by running the experiment in random order, you build in protection against trends in response biasing the results.

Explore what to do if a point falls out of the limits: If there were an outlier, you could click on it to get the

coordinates displayed to the left of the graph. The program remembers the point. It will remain highlighted on other

plots. This is especially helpful in the residual analysis because you can track any suspect point. This feature also works

in the interpretation graphs. Give it a try! Click anywhere else on the graph to turn the point off.

Even more helpful may be the option to highlight a point as shown below.

Run Number

Ex

tern

all

y S

tud

en

tiz

ed

Re

sid

ua

ls Residuals vs. Run

-4.00

-2.00

0.00

2.00

4.00

1 4 7 10 13 16


Highlighting a point

This flags the run back in your design layout and elsewhere within Design-Expert so you can keep track of it. Check it

out!

Skip ahead to the Box Cox plot. This was developed to calculate the best power law transformation. (Refer to Montgomery’s Design and Analysis of Experiments textbook for details.) The text on the left side of the screen gives the recommended transformation, in this case “None.” That’s all you really need to know!

Explore some details on Box-Cox: For those of you who want to delve into the details, note that the Box-Cox screen

is color coded to help with interpretation. The blue line shows the current transformation. In this case it points to a value

of 1 for “Lambda,” which symbolizes the power applied to your response values. A lambda of 1 indicates no

transformation. The green line indicates the best lambda value, while the red lines indicate the 95% confidence interval

surrounding it. If this 95% confidence interval includes 1, then no transformation is recommended. It boils down to this:

If the blue line falls within the red lines, you are in the optimal zone, so no change need be made in your response

transformation (in this case – none).

Box Cox plot for power transformations

P.S. The Box Cox plot will not help if the appropriate transformation is either the logit or the arcsine square root

transformation. See the program’s Help system write-up on “Response Transformations” for further details.

Go back to the Diagnostics Tool, press the Influence button and then Report. Here you see the numerical values for diagnostic statistics reported case-by-case. Discrepant values will be flagged. In this case nothing is detected as being abnormal.



Diagnostics report (only partially show in this screen shot)

Explore other measures of influence: Go back and look at the leverage, DFFits, DFBetas and Cook’s distance

diagnostics on this side of the Design-Expert tool. Click the Tips light bulb for enlightenment. ; )

Examine Main Effects and Any Interactions

Assuming that the residual analyses do not reveal any problems (no problems are evident in our example), it’s now time to look at the significant factor effects.

On the analytical tool bar at the top of the screen, choose the Model Graphs tab. The AC interaction plot comes up by default. (If your graph displays the x-axis in coded units, return to actual units by choosing Display Options, Process Factors - Actual.)

Interaction graph of factors A (temperature) versus C (concentration)

The “I-Beam” symbols on this plot (and other effect plots) depict the 95% least significant difference (LSD) interval for the plotted points.


Explore how to get the LSD value: To view a numerical value for LSD, click any of the points (squares or triangles

representing predicted outcome). Try it!

P.S. If the “I-Beam” symbols do not appear, right-click within the graph’s boundary, then choose Graph Preferences,

XY Graphs tab. Under Factorial Graphs, click on Use LSD Bars.

Those points that have non-overlapping intervals (i.e. the LSD bars don’t intersect or overlap from left to right through an imaginary horizontal line) are significantly different.

Explore how to superimpose a reference line: An easy way to verify separation is to draw a horizontal reference line.

Right-click on the graph near the top of the left red LSD bar (C+ at 24 degrees) and choose Draw Horizontal Reference.

This will put a blue reference line on the graph as shown above. Drag it down to the position displayed.

Reference line added to graph

The reference line makes it clear that the left red LSD (C+ at 24 degrees) overlaps with the right red LSD (C+ at 35

degrees), thus there is not a statistically significant difference between those two predictions.

Note also that the spread of the points on the right side of the graph (where Temperature is high) is smaller than the spread between the points at the left side of the graph (where Temperature is low.) In other words, the effect of formaldehyde concentration (C) is less significant at the high level of temperature (A). Therefore, the experimenters can go to high temperature and reduce the concentration of harmful formaldehyde, while maintaining or even increasing filtration rate. This combination is represented by the black square symbol at the upper right of the interaction plot.

The floating Factors Tool palette opens along with the default plot. Move the floating palette as needed by clicking the top blue border and dragging it. This tool controls which factor(s) are plotted on the graph. At the bottom of the Factors Tool is a pull-down list from which you can also select the factors to plot. Only the terms that are in the model are included in this list.

Explore what happens if you pick a main effect that’s involved in interaction: Click the Term list down-arrow and

select A. Notice that the Graphs Tool shifts from Interaction to One Factor.

Changing to the main effect of A (a one-factor plot)

More importantly, pay heed to the warning at the top of the plot of A (Temperature). It states “Factor involved in an

interaction.” You should never try to interpret main effects plots of factors involved in interactions because they provide

misleading information.



Let’s do something more productive at this stage: Go back to the Factors Tool and select from the Term list the other significant interaction AD.

Interaction AD

Notice on the Factors Tool that factors not already assigned to an axis (X1 or X2) display a vertical red slider bar. This allows you to choose specific settings. The bars default to the midpoint levels of these non-axes factors.

Explore the factor slider tool: You can change levels of fixed factors by dragging the bars, or by typing the desired

level in the numeric space near the bottom of the Factors Tool. Check this out by grabbing the slide bar for factor

C:Concentration and moving it to the left. Notice how the interaction graph changes.

Interaction AD with the slider bar for factor C set far left at its low (–) level

It now becomes clear that a very high filtration rate can be achieved by going to the high stir rate (the red line for factor

D). Click this high point to get all the details on its response and factor values. This is the optimum outcome.

To reset the graph to its default concentration, you could type “3”at the bottom of the Factors Tool. (Factor C must be

clicked and highlighted for this to take effect.) You can also get the original settings back by pressing the Default


button. Give it a try, but remember that going to low concentration of formaldehyde was a primary objective for this

process-troubleshooting experiment, so be sure to slide C back to the left again.

P.S. Yet another option is available for setting factor levels, which you can see by going to the Factors Tool and clicking

the Sheet button. This view offers alternative modes for specifying how to set up your plot. In the columns labeled

“Axis” and “Value” you can change the axes settings or type in specific values for factors. This will come in handy!

Sheet View of Factors Tool

Click back to Gauges to return the Factors Tool to its default configuration.

You can view the AD interaction with the axes reversed by right-clicking D:Stir Rate and changing it to X1 axis.

Axes switched

It makes no difference statistically, but it may make more sense this way.



Explore how to change text: One last thing: You can edit at least some text on many of your graphs by right-clicking

your mouse. For example, on the interaction graph you can right-click the X1-axis label. Then choose Edit Text. The

program then provides an entry field. Try it!

Edit text capability

Draw the Cube Plot

Now from the floating Graphs Tool select Cube to see the predicted response as a function of the three factors that created significant effects: A, C, and D.

Cube plot of A, C, and D

This plot shows how three factors combine to affect the response. All values shown are predicted values, thus allowing plots to be made even with missing actual data. Because the factors of interest here are A, C, and D, the program picked them by default. (You can change axes by right-clicking any factor on the Factors Tool.)

Filtration rate is maximum at settings A+, D+, C- (lower back right corner with predicted response over 100), which also corresponds to the reduced formaldehyde concentration. Fantastic!!

Produce Contour and 3D Plots of the Interaction

An interaction represents a non-linear response of second order. It may be helpful to look at contour and 3D views of the interaction to get a feel for the non-linearity.

First select Contour to get a contour graph. The axes should come up as A (Temperature) and C (Concentration). If not, simply right click over the Factor Tool and make the appropriate changes.


Contour graph

You may be surprised to see the variety of colors – graduated from cool blue for lower response levels to warm yellow for higher values.

Explore how to add flags, add or change contours, etc.: Design-Expert contour plots are highly interactive. For

example, you can click on a contour to highlight it. Then you can drag it to a new location. Furthermore, by right-

clicking anywhere within the graph you can bring up options to add flags, add contours, or change graph preferences.

Adding a flag (via right-click menu)

If you like, play around a bit with these options.

Now, to create an impressive-looking graph that makes it easy to see how things shape up from this experiment, select 3D Surface.



A 3D view of the AD interaction (with flag set)

Move your mouse cursor over the graph. When it turns into a hand () press down and rotate the view however you like. It’s fun! Double click to re-set the graph to its original coordinates.

Explore another way to rotate graphs: Select View, Show Rotation.

Pulling up the Rotation wheel

Grab the wheel with your mouse and rotate the plot to different angles. Press the Default button to re-set the graph to its

original coordinates.

Before moving on to the last stage, take a look at the first interaction by going to the Factors Tool and from the Term list selecting AC. Move the slide bar on D: Stir Rate to the right (maximum level) to increase the response.


3D plot for first interaction: AC

We have nearly exhausted all the useful tools offered by Design-Expert for such a basic design of experiments, but there’s one more tool to try.

Confirmation

The last feature that we will explore appears at the lower left of the screen under the main branch of Post Analysis: Click the Confirmation node to make response predictions for any set of conditions for the process factors. When you first enter this screen, the floating Factors Tool palette defaults to the center points of each factor. Levels are easily adjusted with the red slider bars or more precisely by using the Sheet view. In this case, the analysis suggests that you should slide the factors as follows:

A (Temperature) right to its high level (+)

B (Pressure) leave at default level of center point

C (Concentration) left to its low level (-)

D (Stir Rate) right to its high level (+)

Confirmation at best factor settings

Design-Expert uses the model derived from experimental results to predict the mean and interval (PI) for the number (n) of confirmation runs at the conditions you dialed up, which provide the highest filtration rate with the least amount of formaldehyde. This is mission accomplished!



Explore other features of Post Analysis: To provide greater assurance of confirmation it makes sense to increase the

sample size (n). The returns in terms of improving the prediction interval (PI) diminish as n increases. You can see this

for yourself by trying different values of n in the Confirmation: Watch what happens to the PI—it approaches a limiting

value.* Running six or so confirmation runs may be a reasonable compromise.

As shown below, you can create a date entry screen to log in your confirmation runs. Design-Expert then calculates the

mean for the actual results. If this falls within the PI, you can press ahead to the next phase of your experimental

program. Otherwise you must be wary.

Entering confirmation results

*Statistical detail: For an infinite number of confirmation runs the PI converges to the confidence interval (CI) which is a

function of the number (N) of runs in the original experiment. See the CI in the Point Prediction node. There you also

find the tolerance interval (TI) calculated to contain a proportion (P) of the sampled population with a confidence (1-).

By default, the interval contains 99% of the sampled population with 95% confidence. Note that the 95% confidence

interval on a mean will have the narrowest spread, with the 95% prediction interval for a single observation (given in the

Confirmation node) being wider and the 95% TI which contains 99% of the population being the widest. The most

rigorous interval, TI, is often required for setting manufacturing specifications, but most experimenters will settle for the

PI as a way to ‘manage expectations.’ Thus the Confirmation tool will be your best friend for moving forward.

You’ve now viewed all the important outputs for analysis of factorials. We suggest you do a File, Save at this point to preserve your work. Design-Expert will save the model you created so the outputs can be quickly reproduced if necessary. Remember that if you want to write some comments on the file for future identification, you can click the Notes folder node at the top left of the tree structure at the left of your screen and then type in the description. It will be there to see when you re-open the file in the future.

Prepare Final Report

Now all that remains is to prepare and print the final reports. If you haven’t already done so, just click the appropriate icon(s) and/or buttons to bring the information back up on your screen, and click the print icon (or use the File, Print command).

You can also copy graphs to other applications: Use Edit, Copy or press the copy icon as shown below.


Copy icon

For ANOVA or other reports be sure to do a Select All first, or highlight the text you wish to copy.

Explore options for exporting results: Try right clicking over various displays within Design-Expert software. It

often provides shortcuts to other programs that facilitate your reporting. For example, see the screenshot below taken

from the design layout. (Tip: To highlight all of the headers and data, click the Select button.)

Options for export

If you’ve made it this far and explored all the sidebars, then you know that the best way to learn about features is to be

adventurous and try stuff. But don’t be too stubborn on learning things the hard way: Press screen tips for advice and go

to the main menu Help and search there as well.

This completes the basic tutorial on factorial design. Move on to the tutorials on advanced topics and features if you like, or exit from Design-Expert by choosing File, Exit from the menu. If you have not stored your data, or you made changes since the last save, a warning message will appear. Exit only when you are sure that your data and results have been stored.

DX9-03-2-Foldover-V8.docx Rev. 5/2/14

Design-Expert 8 User’s Guide Fractional Factorial with Foldover Tutorial 1

Fractional Factorial with Foldover Tutorial

Introduction

At the outset of your experimental program you may be tempted to design one comprehensive experiment that includes all known factors - to get the BIG Picture in one shot. This assumes that you can identify all the important factors and their optimal levels. A more efficient, and less risky, approach consists of a sequence of smaller experiments. You can then assess results after each experiment and use what you learn for designing the next experiment. Factors may be dropped or added in mid-stream, and levels evolved to their optimal range.

Highly fractionated experiments make good building blocks for sequential experiments. Many people use Plackett-Burman designs for this purpose, but we prefer the standard two-level approach or the minimum-run (“Min Run”) options offered by Design-Expert® software. Regardless of your approach, you may be confounded in the interpretation of effects from these low-resolution designs. Of particular concern, main effects may be aliased with plausible two-factor interactions. If this occurs, you might be able to eliminate the confounding by running further experiments using a foldover design. This technique adds further fractions to the original design matrix.

We will discuss foldover designs that offer the ability to:

free main effects from two-factor interactions, or

de-alias a main effect and all of its two-factor interactions from other main effects and two-factor interactions.

Design-Expert’s “Foldover” feature automatically generates the additional design points needed for either type of foldover.

Saturated Design Example

Lance Legstrong has one week to fine-tune his bicycle before the early-bird Spring meet. He decides to test seven factors in only eight runs around the quarter-mile track. We show Lance’s design below, which can be set up from Design-Expert via its factorial tab with the default design builder for a “2-Level Factorial”. It is “saturated” with factors, that is, no more can be added for the given number of runs. (Inspiration for this case study came from Statistics for Experimenters, 2nd Edition (Wiley, 2005) by Box, Hunter and Hunter (“BHH”). In Section 6.5, BHH reports a very similar study with a few factors and results differing from those shown below.)

“When your television set misbehaves, you may discover that a kick in the right place often fixes the problem, at least temporarily. However, for a long-term solution the reason for the fault must be discovered. In general, problems can be fixed or they may be solved. Experimental design catalyzes both fixing and solving.” -- Box, Hunter and Hunter.

2 Fractional Factorial with Foldover Tutorial Design-Expert 8 User’s Guide

Std Run A: Seat

B: Tires psi

C: Handle Bars

D: Helmet Brand

E: Gear

F: Wheel covers

G: Gener-

ator

Time 1/4

mile secs

1 2 Up 40 Up Windy Low Off Off 77

2 1 Down 40 Up Atlas High Off On 74

3 4 Up 50 Up Atlas Low On On 82

4 7 Down 50 Up Windy High On Off 47

5 6 Up 40 Down Windy High On On 72

6 3 Down 40 Down Atlas Low On Off 77

7 8 Up 50 Down Atlas High Off Off 48

8 5 Down 50 Down Windy Low Off On 83

Experimental design and results

As you will see in a moment via an evaluation, this is a resolution III design in which all main effects are confounded with two-factor interactions. (To alert users about these low-resolution designs, they are red-flagged in the design builder within Design-Expert.)

Two-level factorial design builder in Design-Expert

Lance would prefer to run a resolution V or higher experiment, which would give clear estimates of main effects and their two-factor interactions. Unfortunately, at the very least this would require 30 runs via the “min-run Res V” design (also known as “MR5”) invented by Stat-Ease statisticians. But 30 runs are too many for the short time remaining before competition. Lance briefly considers choosing an intermediate resolution IV design (color-coded a cautionary yellow in Design-Expert), but seeing that it requires 16 runs, he decides to perform only the 8 runs needed to achieve the lesser resolution III.

To save time on data entry, select File, Open Design and choose Biker.dxp. After looking over the data, check out the alias structure of the design by clicking the Evaluation node on the left. Note that Design-Expert recognizes that this meager design can only model response(s) to the order of main effects. The program is set



to produce results showing aliasing up to two-factor interactions (2FI) – ignoring terms of 3FI or more.

Evaluating the biker’s experiment

Now press the Results button at the top of the data window to see the impact of only running a Res III design in this case.

Alias structure for first block of biking time-trials

This design is a 1/16th fraction, so every effect will be aliased with 15 other effects, most of which are ignored by default to avoid unnecessary screen clutter. The output indicates that each main effect will be confounded with three two-factor interactions.

Don’t bother studying the remaining design evaluation report. Click the Time ¼

mile node in the analysis branch at the left and Effects at the top of the display window. Starting from the right, click the three largest effects (or rope them off as


pictured below). Notice how the line jumps up and a gap appears between the trivial many effects near the zero effect level and the vital three effects that you picked.

Effect graph – three largest ones selected

On the floating Effects Tool, click Pareto Chart.

Pareto chart selected on Effects Tool

Then on the chart itself, right-click the biggest bar (E) to show its aliases as below.



Pareto chart of effects with aliases displayed for the bar labeled “E”

Factors B (Tires), E (Gear), and G (Generator) are clearly significant. Click ANOVA to test this statistically. Before you can go there, you are reminded via the warning shown below that this design is aliased.

Alias warning

You know about this from the design evaluation, so click No to continue. (Option: click Yes to go back and see the list. Then click the ANOVA button.) The output shown below has the View, Annotated ANOVA option turned off to provide a cleaner look at the statistics.


ANOVA output – unannotated

Lance gets a great fit to his data. Based on the negative coefficient for factor B, it seems that hard tires decrease his time around the track. On the other hand, the positive coefficients for factors E and G indicate that high gear gives him better speed and the generator should be left off. The whole approach looks very scientific and definitive. In fact, after adjusting his bicycle according to the results of the experiment, Lance goes out and wins his race!

However, Lance’s friend (and personal coach) Sheryl Songbird spots a fallacy in the conclusions. She points out that all the main effects in this design are confounded with two-factor interactions. Maybe one of those confounded interactions is actually what’s important. “You must run at least one more experiment to clear this up,” says Sheryl. “That sounds like a great idea,” says Lance tiredly, realizing that he will never sleep easy knowing that some doubt remains as to the ideal settings for his bike.

Complete Foldover Design

To untangle the main effects from the interactions in his initial resolution III design, Lance can run a “foldover,” which requires a reversal in all of the signs on the original eight runs. Combining both blocks of runs produces a resolution IV design – all main effects will be free and clear of two-factor interactions (2FI’s). However, all the 2FI’s remain confounded with each other.

To create the new runs for the foldover design, click the Design node and then select Design Tools, Augment Design, Augment.



Augment the first block of runs

This selection brings up the following screen.

Foldover option suggested by default

The default suggestion by Design-Expert for Foldover is good, so click OK to continue. The program now lists which factors to be folded over. By default, Design-Expert selects all the factors. This represents a complete foldover. Factors can be selected or cleared with a right click on the factor, or by double-clicking on the factor.

Choice of factors to fold over

In this case it is best that all factors be folded over, so click OK to continue. The program displays the following warning (really just a suggestion).


Suggestion to evaluate foldover design

Click Yes and press Results to see the augmented design evaluation – much better!

Aliasing after foldover

Complete foldover of a resolution III design results in a resolution IV design. The proof for this case is that the main effects are now aliased only with three-factor interactions – ignored by default because they are so unlikely. See Montgomery’s Design and Analysis of Experiments textbook for the details on why this happens. By the way, the power also increases due to the doubling of experimental runs. If you’d like to assess this, go back to the Model screen and change the Order to Main effects before re-computing the Results. Then press the Power button on the floating Bookmark tool.

Now return to the main Design node and select View, Display Columns, Sort, Std Order. (Tip: A quicker route to sorting any column is via a right-click on the header.) Notice how each run in block two is the reverse of block one. For example, whereas the seat (factor A) went up, down… in block 1, it goes down, up… in block 2 – everything goes opposite.



Folded-over design in standard order

Lance’s results for the additional runs appear in the following table.

Std Blk A: Seat

B: Tires psi

C: Handle

Bars

D: Helmet Brand

E: Gear

F: Wheel covers

G: Gener-

ator

Time 1/4 mile

secs

9 2 Down 50 Down Atlas High On On 57

10 2 Up 50 Down Windy Low On Off 93

11 2 Down 40 Down Windy High Off Off 84

12 2 Up 40 Down Atlas Low Off On 87

13 2 Down 50 Up Atlas Low Off Off 94

14 2 Up 50 Up Windy High Off On 57

15 2 Down 40 Up Windy Low On On 86

16 2 Up 40 Up Atlas High On Off 83

Experimental results for foldover block

To continue following along with Lance, within your Response 1 column, enter his above times for the foldover block (Block 2) of eight runs, as shown below. First be sure you have changed the design matrix to standard order. Input factors must match up with the responses, otherwise the analysis is nonsense.

Folded-over design in standard order with block-2 times (secs) keyed in


Click the Time ¼ mile node and display the Effects plot. Starting from the right, click the three largest effects (or rope them off as shown earlier).

Half-normal plot of effects after foldover

It now appears that what looked to be effect G really must be AF. Be careful though – recall from the design evaluation that AF is aliased with BE and CD in this folded over design. (To see this, you can go back and look at the screen shot from before or press Evaluation on your screen and look at the Report. Or ,even better, just right click on AF on the Half-Normal Plot (as pictured above)). Design-Expert picks AF only because it is the lowest in alphabetical order of all the aliased terms.

From their subject matter knowledge, Lance and Sheryl know that AF, the interaction of the seat position with the wheel covers, cannot occur; and CD, the interaction of handle bar position with the brand of helmet, should not exist. They are sure the only feasible interaction is BE, the interaction of tire pressure and gears. However, proceed with the ANOVA on the model with the two-factor interaction at the default of AF

The software gives the following warning about model hierarchy.

Warning that model is not hierarchical

Because you selected interaction AF, the parent factors A and F must be included in the model to preserve hierarchy. This makes the model mathematically complete. Click Yes to correct the model for hierarchy.



Hierarchy corrected

Notice that these two new model terms, A and F, fall in the trivial-many near-zero population of effects. Therefore you may find that one or both are not very significant statistically. Nevertheless, it is important to carry them along.

Press forward to ANOVA again. You are now warned that “This design contains aliased terms. Do you want to switch to the View, Alias list…?” Click No to continue. (You will return later to this Alias List, so don’t worry.)

The ANOVA looks good. (Factor A is not significant but it’s in the model to preserve hierarchy.)

ANOVA output after foldover


Click Diagnostics. You will see nothing abnormal here. Then click Model Graphs to view the “AF” interaction plot. Because neither “A” nor “F” as main effects changes the response very much, the interaction forms an X, as shown below.

The AF interaction plot

Keep in mind that Lance and Sheryl know this interaction of the seat position with the wheel covers really cannot occur, so what’s shown on the graph is not applicable.

Investigation of Aliased Interactions

Lance decides to investigate the other, more plausible, interactions aliased with AF. You can follow along on your computer to see how this works. Lance clicks back onto the Effects button. Then, on the floating Effects Tool, he selects Alias List.



Viewing the alias list

He locates and right clicks term AF, below left, then selects BE, resulting in the next screenshot shown.

Substituting BE for AF (before and after)

Note that this action is reversible, that is, AF can be substituted back in for BE. But for now, just to do something different, Lance selects Pareto Chart on the floating Effects Tool. The effects of A and F remain from the prior model in hierarchical support of the AF interaction.


Pareto chart with BE substituted for AF

Now with interaction BE as the substitute, these two main effects, clearly insignificant, are no longer needed, so click the bars labeled “F” (ranked 4th) and “A” (6th) to turn off these vestigial model terms. Notice how this simplifies into a family of effects: B, E and BE. In other words, this model achieves hierarchy with all terms being significant on an individual basis. As a general rule, a simpler (or as statisticians say “more parsimonious”) model like this should be favored over one that requires more terms. This is especially true for models with parent terms that are not significant on their own, such as the one you created earlier with the orphan AF.

Lance now displays the BE interaction. Take a look at it yourself by clicking the Model Graphs button. (Click No if advised to view aliased interactions.)



The BE interaction plot (effect of tires at low versus high gear)

Lance feels that it really makes much better sense this way.

Notice that Design-Expert displays a green point on the graph, which in fractional factorials, only appears under certain conditions. Click it to identify these conditions.

Actual experimental run identified (enlarged circle)

At the left of the graph you now see that at this point the tire pressure (B) equals 40 psi and the gear (E) is set high. Above that you see the run number identified (yours may differ due to randomization) and the actual time of 83 seconds. The other factors, not displayed on the graph, would normally default to their average levels, but because they are categorical, the software arbitrarily chooses their low levels (red slide bars to the left). This setup matches the last row of the foldover in standard order. (Note: due to randomization, your run number may differ from that shown at the left of the graph.)

If you prefer to display the results as an average of the low and high levels, click the list arrow on the floating Factors Tool and make Average your selection, as shown


below (already done for factors A:Seat, C:Handle Bars, D:Helmet Brand, F:Wheelcovers, and G:Generator).

Changing to average levels for factors not displayed in the interaction

Notice that the point on the graph now disappears because the average seat height was not one of the levels actually tested. To return the interaction plot back to original settings, press Default on the Factors Tool. Then play with the other settings. See if you can find any other conditions at which an actual run was performed. Remember that even with the foldover, you’ve only run 16 out of the possible 128 (27) combinations, so you won’t see very many points, if any, on the graphs. For example, as shown below, one of the other actual points (the green circle) can be found with the factors set on the factors tools as shown below (high generator).

Another actual point display by re-setting G: Generator to its high level

Just to cover all bases, Lance decides to look at the second alias for AF – the CD effect, even though it just doesn’t make sense that C (handlebars) and D (helmet brand) would interact. Lance sees that the interaction plots look quite different. What a surprise! If you can spare the time, check this out by going back to the Effects button and selecting to Alias List. Then replace BE with CD via the right-click menu. (You must then select the factors C and D in the model to make it



hierarchical.) Based on his knowledge of biking, Lance makes a “leap of faith” decision: He assumes that BE is the real effect, not CD.

Now would be a good time to save your work by doing a File, Save As and typing in a new name, such as “Biker2”.

Inspired by these exciting results, Lance begins work on a letter advising the bicycle supplier to change how they initially set them up for racers. However, Lance’s friend and personal coach, Sheryl, reminds him that due to the aliasing of two-factor interactions, they still have not proven their theory about the combined impact of tire pressure and gear setting. “You must run a third experiment to confirm the BE interaction effect,” admonishes Sheryl. “I suppose I must,” replies Lance resignedly.

Single Factor Semi-Foldover

Sheryl declares, “By making just eight more runs you can prove your assumption.” “Wonderful,” sighs Lance, “Please show me how.” “Well,” replies Sheryl, “If you run the same points as in the first two experimental blocks, but reverse the pattern only for the B factor, then B and all of its two-factor interactions will be free and clear of any other two-factor interactions.” “But that requires 16 more runs,” Lance complains. “Hold on,” says Sheryl, “I saw an article posted by Stat-Ease on ‘How To Save Runs, Yet Reveal Breakthrough Interactions, By Doing Only A Semifoldover On Medium-Resolution Screening Designs’ at http://www.statease.com/pubs/semifold.pdf . It details how you can do only half the foldover and still accomplish the objective for de-aliasing a 2FI.” Thus, Lance allows himself to be cajoled into another eight times around the track.

You can create the extra runs by returning to the Design node of Design-Expert and from there selecting Design Tools, Augment Design, Augment. The program now advises you do the Semifold.

Augmenting via semifold

Press OK to bring up the dialog box for specifying which runs will be added in what now will become a third block of runs for Lance to ride. In the field for Choose the factor to fold on, select B. Lance and Sheryl puzzle over the next question, Choose the factor and its level to retain, but they quickly settle on D+ – the Windy brand of helmet, figuring that this factor definitely does not affect the response (and Sheryl likes the look of Lance wearing this brand of helmet!).

http://www.statease.com/pubs/semifold.pdf


Specifying the semifold

Give this the OK, click Yes as advised to go to design evaluation, and if the warning about missing data comes up, press Yes again. You should now see the starting screen for Evaluation – press Results to see the new alias structure. Scroll down to view the 2FI’s and notice that BE is now aliased only with three-factor interactions.

Aliasing after the semifold

Notice that some effects, for example AB, are partially aliased with other effects. This is an offshoot of the semifold, causing the overall design to become non-orthogonal, that is, effects no longer can be independently estimated. You will see a repercussion of this when Design-Expert analyzes the data.

Click the main Design branch to see the augmented runs. Then right-click the Std column header and select Sort by Standard Order to get it back in standard order.



Sorting by standard order again

Lance’s results for the third block runs can be seen in the following table.

Std Blk A: Seat

B: Tires psi

C: Handle

Bars

D: Helmet Brand

E: Gear

F: Wheel covers

G: Gener-

ator

Time 1/4

mile secs

17 3 Up 50 Up Windy Low Off Off 98

18 3 Down 40 Up Windy High On Off 88

19 3 Up 50 Down Windy High On On 61

20 3 Down 40 Down Windy Low Off On 92

21 3 Up 40 Down Windy Low On Off 91

22 3 Down 50 Down Windy High Off Off 60

23 3 Up 40 Up Windy High Off On 90

24 3 Down 50 Up Windy Low On On 97

Experimental results for third block (semifold)

Enter the times shown above into your blank response fields. Again, be certain everything matches properly. Then go to the analysis of the Time ¼ Mile and press the Effects button. The following warning comes up that the design is not orthogonal (as explained a bit earlier).

Warning about semifold not being orthogonal

Press OK. Then, starting from the right, click the three largest effects (or rope them as pictured below). You will find B, E, and interaction BE to be significant.


Half-normal plot after semifold

Now on the Effects Tool press Recalculate.

Recalculating effects

The effects may shift very slightly – you may not notice the change. There is an option to automatically recalculate the graph upon selection. Right click within the graph border and click Graph Preferences. Click in the Recalculate effects on

selection box so the checkmark appears (shown below). If the checkmark already appears (which is the default), your graph automatically recalculated so you won’t see any change when clicking recalculate.

Automatically recalculate effects option



When you press ANOVA, the “This design contains aliased terms” warning again appears. Click Yes to see the alias list, which now looks much better than before: The model (“M”) terms are aliased only with high-order interactions that are very unlikely to have any impact on the response.

Alias list brought up by software

Click ANOVA again and review the results. Then move on to Diagnostics. Check out all the graphs. Then click Model Graphs. The interaction plot of BE looks the same as before. Check it out. Just to put a different perspective on things, on the Factors Tool, right click E: Gear and select it for the X1 axis. You now get a plot of EB, rather than BE. In other words, the axes are flipped. Note that the lines are now dotted to signify that gear is a categorical factor (high or low). On the BE plot the lines were solid because B (tire pressure) is a numerical factor, which can be adjusted to any level between the low and high extremes.

Interaction BE with gear chosen for X1 axis


Conclusion

“Thank goodness for designed experiments!” exclaims Lance. “I know now that if I am in low gear then it doesn’t matter much what my tire pressure is, but if I am in high gear then I could improve my time with higher tire pressure.”

“Or,” cautions Sheryl, “You could say that at low tire pressure it doesn’t really matter what gear you are in, but at high pressure you had better be in high gear!”

“Whatever,” moans Lance.

“Another thing I noticed,” prods Sheryl, “Your average time for each set of eight runs increased a lot. It was 70 seconds in the first experiment, 80 in the second set and almost 85 seconds in the last set of runs. You must have been getting tired. It’s a good thing we could extract the block effect or we might have been misled in our conclusions.”

“It doesn’t get any better than DOE,” agrees Lance.

In this instance, by careful augmentation of the saturated design through the foldover techniques as suggested by Design-Expert software (defaults provided via augmentation tool), Lance quickly found the significant factors among the seven he started with, and he pinned down how they interacted. He needed only 24 runs in total – broken down into three blocks. A more conservative approach would be to start with a Resolution V design, where all main effects and two-factor interactions would be free from other main effects and two-factor interactions. But this would have required 30 experimental runs at the least with the MR5 design (offered only by Stat-Ease!), or far more (64) with the standard, classical two-level fractional factorial design (27-1). And lest you forget, Lance Legstrong may not have won the early-bird Spring meet without the results from his first eight-run experiment.

In this case, Lance discovered a winning combination with the saturated Resolution III design. Thus, he developed a fix for the problem via design of experiments. However, without the prompting of Sheryl Songbird, his friend and personal coach, Lance may not have done the necessary follow up experiments via foldover to determine exactly what caused the improvement. She helped him use DOE to solve the problem.

DX9-03-3-Taguchi-V8.docx Rev. 5/2/14

Design-Expert 8 User’s Guide Taguchi Design Tutorial 1

Taguchi Design Tutorial

Introduction

Taguchi’s orthogonal arrays provide an alternative to standard factorial designs. Factors and interactions are assigned to the array columns via linear graphs. For example, look at the first of 18 linear graphs for the Taguchi L16 (16 run two-level factorial).

1

2

3

6

4

5

13

7

8

10

11

149

15

12

A

B

C

F

D

E

N

G

H

K

L

OJ

P

M

First linear graph for L16 array

The figure at upper left displays 15 column numbers available for effect estimation. To the right you see the corresponding factor letters. Starting at the top and going counter-clockwise, you can see that factor C is connected to AB, implying confounding of factor C with the AB interaction. Factor F is connected to BD, and so forth. These relationships describe aliasing for a handful of the possible relationships. The complete alias structure for the L16, generated by Design-Expert® software, is shown below.

[A] = A - BC - DE - FG - HJ -KL - MN - OP

[B] = B - AC - DF - EG - HK -JL - MO - NP

[C] = C - AB - DG - EF - HL - JK - MP - NO

[D] = D - AE - BF - CG - HM - JN - KO - LP

[E] = E - AD - BG - CF - HN - JM - KP - LO

[F] = F - AG - BD - CE - HO - JP - KM - LN

[G] = G - AF - BE - CD - HP - JO - KN - LM

[H] = H - AJ - BK - CL - DM - EN - FO - GP

[J] = J - AH - BL - CK - DN - EM - FP - GO

[K] = K - AL - BH - CJ - DO - EP - FM - GN

[L] = L - AK - BJ - CH - DP - EO - FN - GM

[M] = M - AN - BO - CP - DH - EJ - FK - GL

[N] = N - AM -BP - CO - DJ - EH - FL - GK

[O] = O - AP - BM - CN - DK - EL - FH - GJ

[P] = P - AO - BN - CM - DL -EK - FJ - GH

Aliasing of main effects with two-factor interactions for L16 (first linear graph)

The underlined effects are the aliases revealed by Taguchi’s first linear graph.

The second of Taguchi’s 18 linear graphs is given below.

2 Taguchi Design Tutorial Design-Expert 8 User’s Guide

9

10

3

6

12

5

13

15

8

2

11

141

7

4

J

K

C

F

M

E

N

P

H

B

L

OA

G

D

Second linear graph for L16

This second linear graph reveals the underlined, italicized effects shown below.

[A] = A - BC - DE - FG - HJ - KL - MN - OP

[B] = B - AC - DF - EG - HK - JL - MO - NP










[M] = M - AN - BO - CP - DH -EJ - FK - GL

[N] = N - AM - BP - CO - DJ - EH - FL - GK


[P] = P - AO - BN - CM - DL - EK - FJ - GH

Aliasing of main effects with two-factor interactions for L16 (second linear graph)

In theory, you could build the entire alias structure by going through all 18 linear graphs. But why bother? The complete alias structure is given by Design-Expert software via its Design Evaluation tool.

Case Study

To see how Design-Expert software handles Taguchi arrays, let’s look at a welding example out of System of Experiment Design, Volume 1, page 189 (Quality Resources, 1991). The experimenters identified nine factors (see table below).



Factor Units Level 1 Level 2

Brand J100 B17

Current amps 150 130

Method weaving single

Drying none 1 day

Thickness mm 8 12

Angle degrees 70 60

Stand-off mm 1.5 3.0

Preheat none 150 deg C

Material SS41 SB35

Factors for welding experiment

The experimenters wanted estimates of several interactions: AB, AD, and BD. Looking at the first L16 linear graph (reproduced in part below), we see that column C can be used to estimate the AB interaction, column E to estimate AD, column F to estimate BD, and column O to estimate AP. Taguchi used columns M and N to estimate error.

1

2

3

6

4

5

14

15

A

B

C

F

D

E

O

P

Subset of first linear graph for L16 on welding

The factor assignments are summarized below.

Column Factor Column Factor

A Brand

B Current J Angle

C AB K Stand-off

D Method L Preheat

E AD M

F BD N

G Drying O AP

H Thickness P Material

Factor assignments for L16 on welding

Columns M and N (blank) will be used to estimate error. (Note: there is no column labeled “I” in this or other designs because this is reserved for the intercept of the predictive model.)



Let’s build this design. Choose File, New Design off the menu bar. (The blank-sheet icon on the left of the toolbar is a quicker route to this screen. If you’d like to check this out, press Cancel to re-activate the tool bar. But the fastest way, new to DX8, is to click New Design on the software’s opening page.) Then from the default Factorial tab click Taguchi OA (orthogonal array) and choose L16(2^15) from the pull down menu.

Selecting the Taguchi orthogonal array (OA)

Click the Continue button. The software then presents the alias structure for the chosen design. Notice that C is aliased with AB, E is aliased with AD, F is aliased with BD, and O is aliased with AP. Also, Design-Expert reserves the letter “I” for the intercept in predictive models, so it skips from factor “H” to “J.”



[A] = A - BC - DE - FG - HJ - KL - MN - OP

[B] = B - AC - DF - EG - HK - JL - MO - NP










[M] = M - AN - BO - CP - DH - EJ - FK - GL

[N] = N - AM - BP - CO - DJ - EH - FL - GK


[P] = P - AO - BN - CM - DL - EK - FJ - GH

Alias structure for L16 two-level design (215)

Click the Continue button. On all other designs you would now be prompted to enter factor names. However, for Taguchi designs this will be done later – after you generate the runs layout. Design-Expert now shows the response screen. Enter the 1 response name as “Tensile” and the units as “kg/mm^2”.

Response entry

At this point you can skip the remainder of the fields – used for calculating the power of your design – and continue on. However, it’s best to gain an assessment of the power of this Taguchi design. Assume that it is beneficial to increase weld tensile strength by at least 1 unit on average, and that quality control data generates a standard deviation of 0.5. Enter these values as shown below so Design-Expert can compute the signal to noise ratio – for this design: 2.

Optional power wizard – necessary inputs entered


Press Continue to see the calculated power, which in this case far exceeds the recommended level of 80 percent – the probability of seeing the desired difference in one main effect (an assumption made by Design-Expert for resolution III designs like this).


Click Continue to accept these inputs and generate the design layout window. Design-Expert now displays the experimental runs in random order. Right click the Std column and choose Sort by Standard Order to see Taguchi’s design order.

Sorting design by standard order

Next, right click the factor A heading and choose Edit Info: Enter “Brand” as the name and “J100” and “B17” as level 1 and level 2. These are nominal (named) contrasts, so leave that option as the default. Click OK.

Using edit factor info screen



The remaining factors can be entered in the same way, but to save time, read in the response data via File, Open Design from the main menu. Select the file named Taguchi-L16.dxp. (If you’re asked to save MyDesign.dez, click No.)

Recall that several of the columns (C, E, F, and O) are being used to estimate inter-actions. Two others (M and N) are being used to estimate error. We can delete all these columns and fit the interactions directly. This greatly simplifies the analysis.

To do this, right click the header for column O (Factor 14) to bring up the menu shown below. Choose Delete Factor and confirm with a Yes.

Right-click menu for factor column in design layout

When deleting factors, the letters associated with the remaining factors change, so to avoid confusion, always start at the right and work left. All this work will eventually pay off by putting you in position to take advantage of Design-Expert’s powerful analytical capabilities.

Repeat the Delete Factor operation on the columns for factors N, M, F, E and C. When you finish deleting columns, only nine experimental factors should remain. Right click the Std column and Sort by Standard Order. Your design should now look like that pictured below.

Taguchi L16 after deleting columns so only 9 factors remain

It is helpful to make a table for cross-referencing the original assignment of factor letters with the new (condensed) list.


Original (9) New (9) Discarded (6)

A: Brand A: Brand K: Stand-off

B: Current B: Current L: Preheat

C: AB C: Method M: error

D: Method D: Drying N: error

E: AD E: Thickness O: AP

F: BD F: Angle P: Material

G: Drying G: Stand-off

H: Thickness H: Preheat

J: Angle J: Material

Cross-reference table for factor letter assignments

Note that some interactions also get re-labeled as shown in the table below: AB stays AB, but AD becomes AC; BD becomes BC and AP becomes AJ.

To review alias structure, click the design Evaluation node. For Order select 2FI (two-factor interaction) and click Results. In the related table below, we ignored interactions of three or more factors and underlined two-factor interactions of interest.

Factorial Effects Aliases

[Est. Terms] Aliased Terms

[Intercept] = Intercept

[A] = A - EF - GH

[B] = B - EG - FH

[C] = C - HJ

[D] = D - EJ

[E] = E - AF - BG - DJ

[F] = F - AE - BH

[G] = G - AH - BE

[H] = H - AG - BF - CJ

[J] = J - CH - DE

[AB] = AB + CD + EH + FG

[AC] = AC + BD + GJ

[AD] = AD + BC + FJ

[AJ] = AJ + CG + DF

[BJ] = BJ + CF + DG

[CE] = CE + DH

Alias structure after deleting columns from L16

Notice that all the main effects, plus the four interactions of interest, are aliased with one or more two-factor interactions. The effects now labeled BJ and CE are the two columns used to estimate error, but they too are aliased with two-factor interactions. All of these aliased interactions must be negligible for an accurate analysis.



Analyze the Results

To analyze the results, follow the usual procedure for two-level factorials as demonstrated earlier in the Factorial Design Tutorials. (If you haven’t already done so, go back and complete this tutorial.)

Click the analysis node labeled Tensile, which is found in the tree structure along the left of the main window. Then, click the Effects button displayed in the toolbar at the top of the main window. Click the two largest effects (J and D) on the half-normal plot of effects, or rope them off as shown below.

Half-normal plot of effects

On the floating Effects Tool press Pareto Chart for another view on the relative magnitude of effects. It’s very clear now that factors J and D stand out.


Pareto chart of effects

Click the ANOVA button. Design-Expert now warns you about aliasing and offers a list for you to view.

Warning that design contains aliased terms

Click Yes to see the aliases again – this time with modeled terms identified by “M” and the others labeled “e” for error.



View of alias list

Again click ANOVA. You should now see an annotated ANOVA report by default. If not, select View, Annotated ANOVA. The ANOVA confirms that the effects of D and J are statistically significant.

ANOVA report

Scroll down, or use the handy bookmarks, for more statistics. Then click ahead to the Diagnostics. The diagnostic results don’t look great – for example, the normal


plot of residuals looks a bit off-kilter –but they’re acceptable. Click the Model Graphs button and, if not already picked by default, select factor D as the Term on the floating factors tool. The following graph should now appear.

One-factor plot of the main effect of factor D (drying)

You may be wondering about the circular red symbol. As indicated by the legend to the left of the screen in red, this is an actual design point. Due to the fractional nature of Taguchi designs, you won’t see too many points on the plots of predicted effects. Click any point(s) that do appear to see readouts of their actual response value.

On the floating Factors Tool, click the Term down arrow now and select J (or right-click on the J bar to make it the X1-axis). Notice that Design-Expert defaults to the “lower” level of the categorical factors.

Plotting the second significant main effect (J)

On the factors tool, click the “upper” level (far right point) of D:Drying, which we now know is best for tensile strength. This shifts the red line to the far right, as shown below.



Plot of main effect J (material) with factor D set at high level

The square symbols at either end of the lines show the predicted outcomes. The vertical I-beam-shaped bars appearing through these points represent the least significant difference (LSD) at a 95% confidence level (the default). Click the square symbol (predicted value) at the upper left to get a readout on the LSD (it appears to the left of the graph). Note that this LSD does not overlap through an imaginary horizontal line with the LSD at the lower right. This provides visual verification that the effect shown on the plot is statistically significant.

Maximal point clicked with predicted response noted and LSD reported

This concludes our tutorial on Taguchi orthogonal arrays. Use these designs with caution! Take advantage of Design-Expert’s design evaluation feature for examining aliases. Make sure that likely interactions are not confounded with main effects. For example, in this case, only two main effects (D and J) appear to be significant, but as shown in the alias table, perhaps D is really EJ, and J could be CH and/or DE. Therefore, it would be a good idea to do a follow-up experiment to confirm the effects of D and J.

DX9-04-1-SplitPlotGen.docx Rev. 5/2/2014

Design-Expert 9 User’s Guide Split-Plot General Multilevel-Categoric Factorial Tutorial 1

Split-Plot General Multilevel-Categoric Factorial

Tutorial

Introduction

In some experiment designs you must restrict the randomization. Otherwise it wouldn’t be practical to perform the runs. For example, temperature cannot be easily changed in many applications. The solution may be a “split-plot” design, which originated in the field of agriculture. Experimenters divided large areas of land, called “whole plots,” into smaller “subplots” that could be treated separately. For example, they planted several crop varieties—one each per whole plot, and then applied different fertilizers on each subplot.

The analysis of a split-plot design is tricky, even for statisticians. But Design-Expert® version 9 software makes it easy.

To illustrate how Design-Expert does design and analysis of split plot, let’s follow an example from Montgomery’s Design and Analysis of Experiments. Feel free to skip the boxed bits—these are sidebars for those who want to know all the details and spend more time to explore things.

Explore details on this case study: Refer to Section 14.4 “The Split-Plot Design” in the 8th Edition of the book

referenced above—published by Wiley in 2012—for more details on the experiment’s structure and this particular case.

It’s also covered in Stat-Ease’s computer intensive workshop on Split-Plot Design for Hard-to-Change Factors.

Background for Case Study

A manufacturer wants to determine the effect of three methods of pulp preparation and four levels of cooking temperature on the tensile strength of paper. These twelve combinations can be conducted in one day.

To increase statistical power, the paper chemist decides to perform three replicates of this twelve-run general factorial design over three days, which produces a total of 36 runs. However, it is not feasible to perform the experiment in a random fashion. Instead, on each of the three days, this experimenter produces a batch of pulp and divides it into four samples to be cooked at four different temperatures. The process is repeated for pulp prepared by the two other methods.

A flowchart of the experiment is shown below.

Flowchart of split plot experiment on papermaking process

In statistical terms, the split plot experiment can be structured as:

Whole plots for the three batches of pulp (hard-to-change factor)

Sub-plots for the four samples cooked at four different temperatures (easy to change factor).

200 °F 225 °F 250 °F 275 °F

Pulp a1

200 °F 225 °F 250 °F 275 °F

Pulp a3Pulp a2

200 °F 225 °F 250 °F 275 °F

Day 1, 2 & 3

2 Split-Plot General Multilevel-Categoric Factorial Tutorial Design-Expert 9 User’s Guide

You will set up this design as a blocked (by day) split-plot general factorial.


Let’s build the design. This is done via the usual procedure for general factorials, with days accounted for as blocks. From the Design-Expert welcome screen, click New Design (or click the blank-sheet icon on your toolbar, or choose File, New Design).

Opening a new design

Then from the default Factorial tab under Split-Plot click Multilevel Categoric. Choose 2 as the number of factors. Click the Vertical radio button. Then enter Pulp Prep as your Factor A name. It’s already set up a Hard-to-Change factor so leave that be. Mouse down to the Levels column and type in 3. Then enter a1, a2, and a3 as your L(1), L(2), and L(3) treatment names.

Still on the same screen, name your second factor Temp, with deg F for units, 4 for the number of levels, and 200, 225, 250, and 275 as your treatments. This factor is Easy to Change as already set by the program. For Type select Ordinal, thus denoting its numeric nature.

Your screen should now match that shown below.

Last factor changed to “ordinal”

Press Continue and for Replicate enter 3 and check on () Assign one block per replicate to produce 36 runs (36=3x3x4 levels from the three factors) in 9 groups.



Three replicates with one block per replicate

Again press Continue. Now for Block Names enter Day 1, Day 2 and Day 3.

Block names

Continue and for the one response Name enter Tensile in Units of strength. Then to assess the power of this design, type in Delta of 6 and Sigma of 2.

Entering response name and units, plus the signal to noise ratio

Next click Options and change the Variance ratio to 0.25 and the model Order to 2FI as shown below.

Explore background on variance ratio: Design-Expert defaults to a variance ratio of 1 for group (whole plot) to

residual variance. But in this case, the whole-plot factor, although hard to change, can be controlled with little variation

relative to the sub-plot factor of temperature.

P.S. The changing of the order presumes that the experimenters must be interested in the interaction of the pulp prep with

temperature, otherwise why bother doing it at all. Therefore they want to know if this model term (aB) can be estimated

with a ‘decent’ (80% being generally acceptable) level of power.

Changing options for power calculation


Press Continue and review the power for the design. Notice that, due to the restriction in randomization in the split plot, the power for seeing effects of the hard-to-change factor “a” (Pulp prep) is reduced, whereas the power for see the effects of the easy-to-change factor “B” (Temperature) and its interaction “aB” are increased, albeit only slightly.

Explore a tip about grouping effects from split plots: Due to the structure of this design, interactions of hard to

change (whole plot) factors with ones that are easy to change (sub-plot) must be tested against the sub-plot error.

Results for power

Notice the loss in power for the hard-to-change factor “a” (Pulp Prep) still leaves it above the recommended minimum. Continue to complete the design-building wizard. Design-Expert now displays your design in groups by level of the hard-to-change factor “a” (Pulp prep), within which the easy-to-change factor “B” (Temperature) is laid out in completely random order.

Design built for split plot (run order randomized so your screen probably differs from this)

At this stage the design would normally be printed as a run-sheet for the actual experiment, with resulting responses of tensile strength recorded.



Analyze the Results

To avoid the trouble of typing in data, simply read in the responses via the icon or via File, Open Design from the main menu.

Opening the design file to get the response data

Select the file named Pulp.dxpx. If this file does not appear, download the tutorial files by going to the Help Menu—Tutorials.

The analysis of a split plot design must be done differently that a completely randomized experiment. To get the correct statistical tests for each, Design-Expert creates two separate ANOVA’s for:

whole plot treatment (pulp prep), and

subplot treatment (temperature) and the interaction between whole plot and subplot individually.

Then it fits the full model in order to get meaningful diagnostics and model graphs.

Start by clicking the Analysis node labeled R1:Tensile which can be found in the tree structure along the left of the main window. Then press ahead to the Model tab displayed in the progressive toolbar. Design-Expert preselects terms for the designed-for two-factor interaction (2FI) model—these are labeled “M”. Enable the program to check if any terms can be safely taken out by changing the Selection to Backward as shown in the screen shot below.

Model selection

Any terms that exceed the “Alpha out” criterion (p-value of 0.1 by default) will be removed and re-designated to Error (“e”) f after looking over the statistics, you want to remove a term, simply go back and double-click it (or right-click for a menu of designation choices).

Heads-up on effects selection for general-factorial split plots: Due to the structure of these designs, the handy

graphical half-normal plot of effects is not produced like it would be for an experiment with factors only at two levels.

Instead, run the backward selection algorithm—details for which can by clicking Screen Tips (the light-bulb).

Continue to the ANOVA (REML) tab and review the p-values for significance. In this case all the designed-for model terms survive the elimination process, that is, they are all statistically significant.


A note about terminology: As you probably know, ANOVA is the acronym for analysis of variance—the

fundamental approach to assessing model significance for all DOE where runs can be completely randomized. However,

in a split plot the components of variance are estimated by a different approach called REML, which stands for restricted

maximum likelihood estimation. Sometimes, depending on the structure of the design, REML agrees with ordinary-

least-squares ANOVA exactly, but often the results differ somewhat.

Results of ANOVA (REML)

Details on statistics presented in ANOVA (REML) report: Search Help, Contents for much more information,

including references, on everything you see presented on this screen. Since, in the interest of keeping things to the point,

these tutorials only provide the highlights, we must move on.

Move on to the next phase of analysis—verifying the statistical assumptions—by pressing the Diagnostics tab. First up is the normal plot of the residuals—internally studentized to provide the most effective view of any obvious discrepancies. There are no obvious irregularities in this case—the points fall in line as expected from normal data.

Diagnosing the normal plot of residuals

Examine all the graphs available via the Diagnostics tool. You should see nothing abnormal.



Heads-up on diagnostics for split plots: Due to the structure of these designs, the handy Box-Cox plot for response

transformations us not produced like it would be for a fully-randomized experiment. Also, as you may have noticed, the

option for externally studentizing residuals does not appear. Nor do other deletion diagnostic tools such as Cook’s

Distance, DFFits and DFBetas. These are prices to pay for accommodating hard-to-change factors in an experiment

design.

Press ahead to Model Graphs. On the Factors Tool right click over the C: Temperature bar and change it to the X1 axis.

Changing axes

You now see an interaction plot of the continuous factor temperature at three discrete levels of pulp preparation. From this graph we can see pulp preparation B2 (symbolized by the green triangles) will maximize tensile strength. Evidently it will be most robust to temperature changes, especially in the temperature range of 225 to 275 degrees F.

Conclusion

This is an ingenious application of designed experimentation using a split plot structure to handle the hard-to-change pulp preparation. Design-Expert provides valuable insights on how power is reduced by restricting randomization. In this case the actual effect of pulp prep (factor “a”) far exceeded that of minimal importance so the experiment succeeded.

Z:\Manual\DX9\DX9-04-2-SplitPlotFact-FT.docx 5/2/2014 9:35:00 AM

1 | P a g e DX9 Split-Plot Tutorial

Split-Plot Tutorial for Design-Expert® Software Version 9 (DX9)

Introduction Very often, experimenters set up two-level factorial designs with the best intentions of running them in random

order, but they find that a given factor, such as temperature, cannot be easily changed. In this case, the analysis

should be done by the split-plot method.

Split-plot designs originated in the field of agriculture where experimenters applied one treatment to a large

area of land, called a “whole-plot,” and other treatments to smaller areas of land within the whole-plot—called

“subplots”. For example, the whole-plot treatment might be fertilizer 1 vs. fertilizer 2, with the subplot

treatment being seed type 1 through 8 (see picture below).

A field sectioned into two whole plots (fertilizer type) and eight subplots (seed type)

Case study: Polymerase Chain Reaction (PCR) This example is based on a polymerase chain reaction—a biochemical technology that amplifies DNA for

diagnosing hereditary diseases and other purposes. Due to equipment

limitations, it is not convenient to fully randomize the treatments, so the

biochemists chose a split-plot design. In this case the whole plots are

actually plates that are subjected to varying conditions of time and

temperature. The subplots fall into the wells within each plate, within

which experimenters can randomly apply the remaining factors.

Building the Design

1) From the Factorial tab Split-Plot section, choose a Regular Two-Level

Factorial design. (To bypass the design build, open the file titled “PCR with data.dxpx”. Rebuild via File, New

Design, clicking Yes to “Use previous design info?”,

to work through the steps on your screen.)

a) Change the total number of factors to 9. This

includes both the whole-plot and subplot

factors.

b) Change the Hard-to-change factors to 3. These

are the three thermocycler (whole-plot)

factors.

c) Choose the hard to change factor fraction. In

this case there is no option other than

accepting the default or Full factorial.

However, for 5 or more factors the program

provides options.



d) Change the number of runs to 256. This specifies a 29-1 factorial design for all 9 factors, which is

resolution IX. Note that the box changes green for any design that is Res V or better, meaning you can

fit main effects and two-factor interactions (2FI). Click Continue to see aliases and Continue again.

2) Note here that the hard-to-change factors are labeled a,b, and c (lowercase), while the easy-to-change

factors are E, F, G, H, and J (uppercase). Enter the

factor names and levels. (To avoid all the typing,

go Back a few screen and Cancel—then open the

file noted earlier and rebuild to this point: Then

the factor information will be prefilled.)

Click Continue.

3) Enter the response (Accept the defaults for delta and sigma). Click Continue. Note that the whole plot and

subplots have different power.

We are only running 8 runs for

the whole-plot factors (a, b & c),

so the power is lower than if it

were a completely randomized design. Click Finish.

4) On the Response 1 – Amplification column header right-click and choose Simulate.

5) Choose Load an existing simulation and click Finish.

6) Find the file called PCR.simx and click Open.

Now you have built the design, ran the experiment (actually, simulated it) and

entered the data. Continue on for instruction for analyzing this split-plot DOE.

Analyzing the Design

On the design layout screen you will see your 256 run design split up into 8 whole-plot groups. Analysis is the

same as for a factorial design in Design-Expert 8, except for one key difference: The subplot and whole-plot

effects are analyzed separately. They each get their own half-normal plot. To get started on analyzing the

Amplification response, just click on the R1-Amplification node.

1) No known transform is needed yet, so click the Sub-plot Effects tab.

2) Select the significant effects (those that stand out to the right) by clicking them as shown below.



3) Click on the whole-plot effects button and click Yes when asked the question about hierarchy. As shown

below, select the significant whole-plot effects not already chosen for hierarchy.

4) Click on the ANOVA (REML) tab. The restricted maximum likelihood (REML) analysis is necessary to properly

identify the significant effects and calculate p-values for split-plot designs. In this case, all the whole-plot

and sub-plot terms are significant at the 0.05 level.

5) Click on the Diagnostics tab and analyze the diagnostics as you normally would. The deletion diagnostics

(i.e. externally studentized residuals, DFFITS, DFBETAS, and Cook’s distance) are not available in split-plot

analyses; nor is the Box-Cox plot for assessing transformations.

The model graphs, numerical optimization and other post-analysis options are all available and work as they do

for all two-level factorial designs in Design-Expert. Explore the graphs for conditions that maximize the

“Amplification” response. Then go to the Optimization Numerical node and ask the program to find the optimal

settings for the polymerase chain reaction (PCR).

DX9-04-3-MultifactorRSM-Opt.docx Rev. 3/6/13

Design-Expert 9 User’s Guide Optimal RSM Tutorial 1

Multifactor RSM Tutorial

(Optimal design) This tutorial details how Design-Expert® software crafts a response surface method (RSM) experiment within an irregular process space. If you are in a hurry to learn only the bare essentials of computer-based optimal design, then bypass the sidebars. However, they are well worth the time spent to explore things.

A food scientist wants to optimize a wheat product cooked at varying times versus temperatures. After a series of screening and in-depth factorial designs, the search for a process optimum has been narrowed to two factors, ranging as follows:

A. Temperature, 110 to 180 degrees C.

B. Time, 17 to 23 minutes.

However, it’s been discovered that to initiate desirable starch gelatinization, time must be at least 19 minutes when temperature is at 110 degrees C—the low end of its experimental range. On the other hand, when the temperature is increased to 180 degrees the starch will gel in only 17 minutes.

Constraint at lower levels of factors

To recap: At the lowest level of A, factor B must be at least 19, while at the lowest level of B, factor A must be at least 180. To complicate matters further, the experimenter suspects that the response surface may be wavy. That is, the standard quadratic model used for response surface methods (RSM) may fall short for providing accurate predictions. Therefore, a cubic model is recommended for the design.

A problem like this can be handled by Design-Expert via its constraint tools and optimal design capability.


Start Design-Expert and initiate the design process by clicking the blank-sheet icon on the left of the toolbar.

Click the Response Surface tab, select Optimal (custom) as the design, and enter the L[1] (lower) and L[2] (upper) limits as shown below.

110 180 A

23

17

B

19

2 Optimal RSM Tutorial Design-Expert 9 User’s Guide

Entering factor levels for optimal design

Create the multilinear constraint

Press the Edit Constraints button at the bottom of the screen.

Edit constraints fields for entering equations

At this stage, if you know the multilinear constraint(s), you can enter them directly by doing some math.

Explore the mathematics for multifactor constraints: Multifactor constraints must be entered as an equation taking

the form of: βL ≤ β1A+ β1B…≤ βU, where βL and βU are lower and upper limits, respectively. Anderson and Whitcomb

provide guidelines for developing constraint equations in Appendix 7A of their book RSM Simplified. If, as in this case

for both A and B, you want factors to exceed their constraint points (CP), this equation describes the boundary for the

experimental region:

A B

A A B B

A LL B LL1

CP LL CP LL

where LL is the lower level. In this case, lower levels are 110 for factor A and 17 for factor B. These factors’ CPs are

180 for factor A and 19 for factor B (the endpoints on the diagonal constraint line shown in the figure below).



Constraint points

Plugging in these values produces this equation:

A 110 B 17

1180 110 19 17

However, the software requires formatting as a linear equation. This is accomplished using simple arithmetic as shown

below.

A 110 B 171

70 2

70 2 1 2 A 110 70 B 17

140 2A 220 70B 1190

1550 2A 70B

Dividing both sides by 2 simplifies the equation to 775 A 35B . This last equation can be entered directly, or it can

be derived by Design-Expert from the constraint points while setting up a RSM design.

Using the Constraint Tool

Rather than entering this in directly, let’s make Design-Expert do the math. Click the Add

Constraint Tool button. It’s already set up with the appropriate defaults based on what you’ve entered so far for the design constraints.

Constraints tool

110 180 A

23

17

B

19

CPA

CPB


As directed by the on-screen instructions, start by specifying the combination, or Vertex in geometric terminology, that must be excluded – shown in red below.

Vertex to be excluded

Click the downlist for vertex A and change it to 110.

Specifying the vertex coordinate for factor A

Then change the Vertex coordinate for B to 17.

Vertex coordinate B specified

The next step indicated by the instructions is to enter in the Constraint Point field the level for B (time) which becomes feasible when the temperature is at its low level of 110. Enter 19 – the time need at a minimum to initiate gelatinization of the starch.

Constraint point entered for factor B

Notice that Design-Expert correctly changed the inequality to greater than (>).

110 180 A

23

17

B

19



Now we turn our attention back to factor A (temperature) and its Constraint Point. It must be at least 180 when B (time) is at its vertex (low) level.

Constraint point set for A

Notice that Design-Expert previously changed the inequality to greater than (>), which is correct.

Press OK to see that Design-Expert calculated the same multilinear constraint equation that we derived mathematically.

Resulting multilinear constraint equation

Press OK to accept this inequality constraint. Then press Continue.

Building an optimal design

Leave the Search option default of “Best” and Optimality on the “I” criterion.

Explore details on optimal design options: Take a moment now to read the helpful information provided on screen,

which provide some pros and cons to these defaults. Note that we recommend the I optimal for response surface designs

like this. Press screen tips (light-bulb icon) for more details about the “Optimal Model Selection Screen.” Feel free to

click the link that explains how Design-Expert uses the “Best” search from coordinate and point exchange. Exit out (X)

of the tips for “Optimal Model Selection Screen.”

Press the Edit Model button to upgrade the default model of quadratic to Cubic. (Recall that the experimenter suspects the response might be ‘wavy’ — a surface that may not get fit adequately with only the default quadratic model.)


Cubic model specified

The defaults for the runs are a bit more than the experimenter would like to run for lack of fit and replicates. To limit the number of runs, change the To estimate lack of fit — from 5 (by default) to 4. Also decrease the Replicates to 4. The experimenter realizes this will decrease the robustness of the design, but decides that runs are too costly, so they need to be limited.

Lack of fit and replicates decreased

Press Continue to the response entry — leave this at the generic default of “R1” (we will not look at the experimental results). Then Continue on to build the optimal design for the constrained process space.

Explore how the program arrives at an optimal design: At this stage, Design-Expert randomly chooses a set of

design points called the “bootstrap.” It then repeatedly exchanges alternative points until achieving a set that meets the

specifications established in the model selection screen.

The optimal design (builds may vary due to random bootstraps)

Because of the random bootstraps, design builds vary, but all will be essentially (for all practical purposes) as optimal. Then to see the selection of points:

right-click the Select button and pick Design ID,

right-click the header for the Id column and Sort Ascending,

right-click the Select button and click on the Space Point Type.



You should now see a list of design points that are identified and sorted by their geometry. Due to the random bootstraps, your points may vary from those shown below.

Design layout sorted by ID with point type shown (your design may vary)

Explore the factor combinations selected by the program: Look over the design and, in particular, the point types

labeled as “vertex”: Is the combination of (110, 17) for (A,B) excluded? It should be – this is too low a temperature and

too short a time for the starch to gel.

P.S. To see how Design-Expert built your design, click the Summary node under the Design branch.

To accurately assess whether these combinations of factors provide sufficient information to fit a cubic model, click the design Evaluation node.

Design evaluation for cubic model

Based on the design specification, the program automatically chooses the cubic order for evaluation. Press Results to see how well it designed the experiment. (We went to View and turned off the Annotated Evaluation option before making the screen shot below, but you may want to keep this informative feature enabled.)


Results of design evaluation (yours may vary)

No aliases are found: That’s a good start! However, you may be surprised at the low power statistics and how they vary by term. This is fairly typical for designs with multilinear constraints. Therefore, we recommend you use the fraction of design space (FDS) graph as an alternative for assessing the sizing of such experiments. This will come up in just a moment.

Explore other aspects of design evaluation: Feel free to press forward to the leverage results and beyond by pressing

for this on the floating Bookmarks. Look over the annotations. Also press for Tips and look in Help for statistical

details.

Press the Graphs tab. Let’s assume that the experimenter hopes to see differences as small as two standard deviations in the response, in other words a signal-to-noise ratio of 2. In the floating tool for FDS Graph enter a d of 2. Press the Tab key to see the fraction of design space that achieves this level of precision for prediction (“d” signifying the response difference) given the standard deviation (“s”) of 1.



FDS graph

Ideally the FDS, reported in the legend at the left of the graph, will be 0.8 or more as it is in this case. This design is good to go!

Explore details on fraction of design space (FDS): A great deal of information on FDS is available at your fingertips

via the screen tips. Take a look at this and click the link to the “FDS Graph Tool.” Exit out of the screen-tips help

screen.

On the floating Graphs Tool press Contour to see where your points are located (expect these to differ somewhat from those shown below).

Standard error plot

Our points appear to be well-spaced and the choices for replication (points labeled “2”) seem reasonable. Select View, 3D Surface (or choose this off the Graphs Tool) to see the surface for standard error.


Response surface for standard error

This design produces a relatively flat surface on the standard error plot, that is, it provides very similar precision for predictions throughout the experimental region.

This concludes the tutorial on optimal design for response surface methods.

Explore what happens if you modify the design: You are now on your own. Depending on which points Design-

Expert picked for your design, you may want to add a few more runs* at the extreme vertices, along an edge, or in other

locations to decrease relatively high prediction variances. If you modify the design, go back and evaluate it. Check the

3D view of standard error and see how much lower it becomes at the point(s) you add. In any case, you can rest assured

that, via its optimal design capability, Design-Expert will provide a good spread of processing conditions to fit the model

that you specify, and do it within the feasible region dictated by you.

*We detailed how to modify the design layout in Part 2 of General One-Factor Tutorial (Advanced Features).

Z:\Manual\DX9\DX9-04-3-SplitPlotOpt-FT.docx 5/2/2014 9:36:00 AM

Optimal Split-Plot Tutorial for Design-Expert® Software Version 9 (DX9)

Introduction This tutorial provides an overview on building an optimal split-plot design. It continues on from the Split-Plot

Tutorial to illustrate how an experimenter might get by with a fraction of the runs via

optimal design. To save time typing, you will work from a ‘starter’ file that provides

prefilled factor names.

Building the Design

1) Open DX9 and, via File on the main menu (or the Open Design icon—the open folder),

follow Open Design to the file titled “PCR as Optimal Build (starter).dxpx”. Open this

and rebuild it via File, New Design by clicking Yes to “Use previous design info?”. You

will now see the factors from the prior feature-tour tutorial.

2) From the Split-Plot section on the Factorial tab, choose an Optimal (custom) design.

This option treats factors as categoric, thus making it more versatile for screening or characterizing a

number of multilevel inputs, such as a variety of materials made by a number of suppliers on several types

of equipment. Press Continue.

3) Review the design settings for this custom build. Design-Expert chooses the model points and group count

based on the number of hard-to-change

factors and the model chosen, which by

default will be the 2FI (two-factor

interaction). For more information, read

the explanations on your screen and get

further details via Screen Tips under Tips

on the main menu (or by simply pressing

the light-bulb icon on your screen).

After studying this, click Continue.

4) Accept the prefilled response (“Amplification”) and defaults for the delta and sigma. Click Continue. Now

the program builds the design, which may take a few minutes.

5) Review the results on power. With the restriction on runs for the whole-plot factors (done to make the

experiment easier on these hard-to-change settings), the power is lower than if it were a completely

randomized design. Click Finish to see the 51-run optimal design split up into 9 groups (whole plots).

This concludes our feature tour on designing an optimal split plot experiment. Go back to the prior tutorial for

the analysis of results from this polymerase chain reaction (PCR). Presumably this optimal option would lead to

similar conclusions in only a fraction of the runs—51 versus 512 for the full factorial. That would be nice!

DX9-05-1-OneFactorRSM.docx Rev. 5/2/14

Design-Expert 9 User’s Guide One Factor RSM Tutorial 1

One-Factor RSM Tutorial

(Part 1 – The Basics)

Introduction

In this tutorial you will get an introduction to response surface methods (RSM) at its most elementary level – only one factor. If you are in a hurry, skip the sidebars. These are intended only for those who want to spend more time and explore things.

Explore basic features of the software: It will be assumed that at this stage you’ve mastered many Design-Expert®

software basic features by completing the preceding tutorials. At the very least you ought to first do the General One-

Factor tutorials, basic and advanced, before starting this one.

The data for this one-factor tutorial, shown below, comes from RSM Simplified (Mark J. Anderson and Patrick J. Whitcomb, 2005, Productivity, Inc., New York: Chapter 1).

x:

Departure

(minutes)

y:

Drive time

(minutes)

0 30

2 38

7 40.4

13 38

20 40.4

20 37.2

33 36

40 37.2

40 38.8

47.3 53.2

Commuting times as a function of when the driver leaves home

The independent (x) variable (factor) is the departure time for Mark’s morning commute to work at Stat-Ease, Inc. Time zero (x=0) represents a 6:30 A.M. start, so for example, at time 40, the actual departure is 7:10 A.M. Mark wants to quantify the relationship between time of departure and the length in minutes of his commute – the response “y.”

Let’s begin by setting up this one-factor RSM experiment in Design-Expert. Be forewarned, we must do some editing of the design to deal with some unplanned events in the actual execution. Fortunately, the software allows for such revising in the experimental design layout and deals with any repercussions in the subsequent analysis.


Start Design-Expert. You will see our handy new quick-start page, which includes the main menu and icon bar. Using your mouse, click New Design.

2 One Factor RSM Tutorial Design-Expert 9 User’s Guide

Quick-start page – New Design selection highlighted

You now see four tabs at the left of your screen. Click the tab labeled Response Surface. Then select One Factor design

Explore other response surface design options: Note that the first two designs on this tab — the Central Composite

and Box-Behnken – do not support experiments on only one factor. Work through the Multifactor RSM Tutorial to

explore program tools from multiple-factor response surface methods.

Your screen should now look like the one shown below.

One factor response surface design

Now enter for factor A the Name, Units, Low, and High inputs as shown below.

Entering factor name, units, and low/high experimental range levels

Mark’s initial theory was that traffic comes in waves. In other words, traffic does not simply increase in a linear fashion as rush hour progresses. Instead, he hypothesized that traffic builds up, backs off a bit, and then peaks in terms of density of cars on the roads into town. Standard RSM designs, such as central composite (CCD) and Box-Behnken, are geared to fit quadratic models (refer to RSM Simplified for math details). Generally this degree of polynomial proves more than adequate for approximating the true response. But in this case, where the response may be ‘wavy,’ we will notch up to the third-order Model labeled Cubic. The model droplist is located near the bottom of your screen.



After selecting a cubic model (center points increased by default to 2)

Notice that the number of runs increases from 7 to 10 after upgrading your model from its default of quadratic. This upgrade includes 2 center points added in by default (versus 1 for linear). Thus it takes only 3 more runs to design for the cubic model.

Press Continue to move on to the response entries. Now enter the Name and Units inputs as shown below.

Response data entry

Press Finish to see the resulting design displayed in randomized run order.

Modify the Design As Actually Performed

Normally you’d now print your screen and use it as a procedure (recipe) sheet. However, to reproduce Mark’s experiment it is helpful to right-click the Factor column header, and from the pop-up menu select Sort Ascending.

To account for Mark’s mistake of awakening very, very late one morning, replace the first Factor entry of 0.00 with his actual departure time of 47.3.

First run re-entered at actual value of 47.3

That tardy start was not Mark’s only mistake. (Evidently Mark is not a morning person!) He also got mixed up somehow on another specified departure – so you must also replace the Factor entry of 26.66 with the actual value of 2. (Strange but true: Mark didn’t adhere to his recipe sheet and left too early that morning.)


Changing the second botched factor level

Again, right-click the Factor column and Sort by This Factor so we can better see how times now line up in this design.

OK, notice how this design specifies some departure times to the one-hundredth decimal place, for example 6.68. This is impractical, so let’s round to the nearest minute: Replace Factor levels 6.68 with 7, 13.34 with 13, and 33.32 with 33 as shown below.

Rounding inconvenient factor levels

That was a lot of work, so now is a very good time to preserve it by selecting File, Save As. Type in the name of your choice (such as Drive time) for your data file, which will be saved as a *.dxp type.

Click Save. Now you’re protected in case of a system crash. Next, enter the response values as shown below.



Entering response data

This is another good time to preserve your work, so click the Save icon on the toolbar.

Saving the response data you’ve entered

The design is done and the experiment completed – not quite as Mark originally planned, but perhaps well enough. We will see.

Explore LOESS fit: Click the Graph Columns node to see a scatter plot of drive times. Then go to View and select

Show LOESS fit to see a locally weighted smoothing.

LOESS fit


To change the bandwidth, move your mouse over the line. When it changes to a cross-hair (+) then right click for the

dialog box. Play around with this to see how bandwidth affects the fit. However, keep in mind that this is more for

visualization purposes—it is completely speculative at this stage. Therefore you had best press on from here for a more

conventional regression modeling.

P.S. This really ought to be called LOWESS (locally weighted scatterplot smoothing). However, the inventor, William

Cleveland, liked “loess” (pronounced ‘low is’) because of its “semantic substance”* being this relates to a deposit of fine

clay or silt that in a cut through the earth appears as smooth curve.

*(Cleveland, William S.; Devlin, Susan J. (1988). “Locally-Weighted Regression: An Approach to Regression Analysis

by Local Fitting,” Journal of the American Statistical Association 83 (403): 596–610.

Analyze the Results

Before we start the analysis, be forewarned that you will now get exposed to quite a number of statistics related to least-squares regression and analysis of variance (ANOVA). If you are coming into this cold, pick up a copy of RSM Simplified and keep it handy. For a good guided tour of these RSM analysis statistics, attend the Stat-Ease workshop titled RSM for Process Optimization. Details on this computer-intensive and hands-on class, including what’s needed as a prerequisite, can be found at www.statease.com. Or simply visit our website to see valuable tips and case studies.

Under the Analysis branch click the node labeled Drive time. Design-Expert displays a screen for transforming the response.

Transformation options

As noted at the bottom of the above screen, in this case the response range is not that great (less than three-fold), so do not bother trying any transformation – it can remain at the default of none.

Explore details on transformations: Before moving on, press the screen tips button (or select Tips, Screen Tips).

This is a very handy help system that tells you about any screen you are viewing. As you travel from one screen to the

next for the first time, keep pressing screen tips to get a brief overview on a just-in-time basis. For more detail, go to

program Help and search on a specific topic.

Now press Fit Summary. Design-Expert provides a summary to start. Let’s look at the underlying tables – start by pressing the Sum of Squares on the floating Bookmarks tool. You then see a table that evaluates each degree of the model from the mean on up.

http://www.statease.com/



Fit Summary – table of sequential model sum of squares

See RSM Simplified Chapter 4 if you are interested in the details. The program suggests the cubic model and underlines that line in this table of sequential sum of squares. The extremely low p-value indicates a highly significant advantage when adding this level to what’s already been built (mean, linear, and quadratic).

Explore options for help: Remember to try the screen tips on this screen. Also, try right-clicking on a given cell to

see if the program offers context-sensitive Help, as it does below.

Accessing context-sensitive Help by right-clicking a report cell

Also, consider referring to program Help via the main menu.

P.S. Notice the options to export output into Word or PowerPoint. This would be a good time to give this a try.

Scroll down to the next section of output, which displays tests for lack of fit.


Lack of fit tests

The cubic model produces insignificant lack of fit – that’s good!

On the floating Bookmark press the R-Squared button to jump down to the last section of the fit summary report – model summary statistics.

Model summary statistics

It should be no wonder that Design-Expert suggests cubic. Look how much lower the standard deviation is from this model and how much better it is compared with lower-order models for R-squared – raw, adjusted, and predicted. Also the cubic model produces the least PRESS (predicted residual sum of squares) – a good measure of its relative precision for forecasting future outcomes.

Move on by pressing the Model tab.



The cubic model chosen

It’s pre-set the way the software suggested, so without further ado, press ANOVA for the analysis of variance. Design-Expert reports that the outcome for the model is statistically significant. It also tells you that the lack-of-fit is not significant.

Analysis of variance (ANOVA)

Press R-Squared on the floating Bookmarks palette and move on to the next section of output, which displays various model statistics.

Post-ANOVA statistics

Explore annotations: Most of these measures have already appeared in the Fit Summary report, but a few are newly

reported. Read the annotations and, if you need more detail, get Help by right-clicking on any particular statistic.

Click Coefficients on the floating Bookmark palette to see details on the model coefficients, including confidence intervals (CI) and the variance inflation factors (VIF) – a measure of factor collinearity. A simple rule-of-thumb is that VIFs less than 10 can be safely disregarded, so perhaps Mark did not botch things too badly by missing some of his scheduled times for departure.


Details on model coefficients

After this you see the predictive equations in terms of coded factor levels and the actuals.

Final equations for predicting drive time

This last formula will be most useful for Mark, because he can simply plug in his departure time in minutes (remember that zero represents 6:30 a. m.) and get an estimate of how long it will take to drive into town for work at Stat-Ease. However, it pays to do some checking before making use of predictive models generated via RSM.

Explore a tool that exports the formula into a spreadsheet: Right click any part of equation to pull up the option for

Copy Excel Formula.

Copying the formula to Excel spreadsheet

Now, if you have this program installed, open Microsoft Excel and Paste. Enter in a departure time and see what’s

predicted. In this example, the equation predicts a 38 minute commute if Mark leaves home 5 minutes beyond his target

time of 6:30 in the morning.

Plug and chug predicts commute time

Analyze Residuals

Press the Diagnostics button to see a normal plot of the residuals.



Normal plot of residuals (longest drive time highlighted)

Notice they are colored by drive time. Click the red one – this is the longest commute resulting from Mark oversleeping one day when the design said he ought to have left at the earliest time.

We could say a lot more about this plot, but let’s just call it good, because all the points line up nicely and the test for departure from normality is insignificant.

Explore how to interpret the normal plot of residuals (and other diagnostics): For more details, press Tips. Also,

refer to preceding tutorial General One-Factor, which delves into the Diagnostic tools of Design-Expert software.

On the floating Diagnostics Tool, press the Resid. vs Run button

Residuals by run (your order may differ due to randomization)

Notice that the highlighted residual — the one stemming from the highest response — falls well within the red lines, that is, it varies only due to common-cause variations. Thus there is no reason to remove this result, albeit unplanned experimentally, from the analysis.


Explore further thoughts on the residuals versus run plot: The first thing to watch for, obviously, would be a single

point falling outside the red line, that is, an outlier. In any case, the decision on whether to keep data or not ultimately

depends on the judgment of the subject-matter expert. In this case, based on a decade of experience commuting daily

into work and confirmation runs after his experiment, Mark chose to keep the point in question (the one highlighted).

It’s simple really: If one leave so late that one gets caught in rush hour, one will spend more time driving!

Another thing to look for in the run plot is trends or shifts. For example, if the winter came midway through a driving

experiment like this, it would probably create a shift. Randomization is vital for protecting against time-related changes

like this that otherwise would bias the outcome. Always randomize your experimental run order.

P.S. We will not explain here why (for good statistical reasons) residuals are “externally studentized.” Other tutorials

might say a few things about this. However, your best resource will be Screen Tips and program Help. If you need

enlightenment, now is a good time to seek out information ‘under the covers’ of Design-Expert.

Now press the Pred. vs Actual button to see a plot showing how precisely the drive time is modeled.

Predicted versus actual response

The points show some scatter around the 45 degree line in the times below 40, but it hits the high point directly. That’s good!

Explore leverage: Not to belabor this, but recall that Mark never intended to leave so late that he’d get into the rush

hour that precipitated such a long drive time. By including the result, he degraded the quality of the original design laid

out by Design-Expert. In particular, the added point is very influential in the fitting. To assess the impact, click the

button labeled Influence to see the second plot on this side of the list — the one for Leverage. (Note: your plot may

differ due to the randomization of run order that Design-Expert changes whenever it rebuilds a design.) You should now

see that the leverage for the longest drive-time point falls above the red-line threshold for this statistic — twice the

average leverage. (A statistical detail: The average leverage (0.4) is simply calculated by dividing the number of model

coefficients (4 including the intercept) by the number of runs (10).)



Leverage (your pattern may differ due to randomization of run order)

This highlighted point is not a statistical outlier — it fell within the limits on the run chart. In fact, Mark (to his chagrin)

observes similarly long drive times whenever he departs late from home. However, this particular set of experimental

data relies heavily on this one point for fitting one or more of the model terms. That’s good to know.

View the Effects Plot

OK, we are finally at the stage where we can generate the response surface plot and see how drive time varies as a function of the time of departure: Press Model Graphs to produce the response surface plot. The dotted lines represent the 95% confidence band on the mean prediction at any given departure time.

One factor model graph (response surface plot)

Oops! The program still thinks Mark will never leave more than 40 minutes later than the earliest time (6:30 A.M.). But as you know, he goofed up one morning and left later. With your mouse over the plot, right-click and select Graph Preferences to remedy this discrepancy between planned and actual maximum factor level.


Graph preferences menu

On the X Axis dialog box, which comes up by default, change the High level to 50.

Graph preference options (via right-click on plot)

Finally, select the Y Axis tab and enter for the Low end a value of 30 and for the High side the level of 70. Also change Ticks to 5 (it will look better this way).

Changing the Y axis range

Click OK to see how this changes the plot. (The warning about the factor value being outside of the design space is a helpful reminder that Mark overslept that one morning and left much later than planned.)



Response plot with X-axis expanded to include highest actual level

Ah ha! It appears that Mark might be seeing a ‘hole’ in the traffic, that is, a trough in the drive time that opens up 25 minutes or more after the earliest departure time. Therefore, he might get a bit more sleep without paying too harsh of a penalty in the form of a longer commute. However, he’d better be careful, further delays from this point could cause him to be very tardy for work at Stat-Ease.

Explore options for exporting graphs: The figure above was produced via Edit, Copy from Design-Expert and then

Edit, Paste to Microsoft Word. As you saw on the Graph Preferences menu, Design-Expert also provides tools for direct

export to Word or PowerPoint. If you have these Microsoft Office applications installed, now is a good time to try these

export options. Also, via Design-Expert’s File menu option you can also Export Graph to file. Design-Expert offers

many Save as options, including encapsulated postscript (“eps”) – popular with publishers of journals and textbooks.

That’s it for now. Save the results by going to File, Save. You can now Exit Design-Expert if you like, or keep it open and go on to the next tutorial – part two for one-factor RSM design and analysis, which delves into advanced features via further adventures in driving.

Design-Expert® SoftwareFactor Coding: ActualDrive time (minutes)

Design Points95% CI Bands

X1 = A: Departure

A: Departure (minutes)

0.00 10.00 20.00 30.00 40.00 50.00

Driv

e t

ime

(m

inu

tes

)

30

40

50

60

70Warning! Factor value outside of design space.

Warning! Interval off graph.

One Factor


One-Factor RSM Tutorial

(Part 2 – Advanced topics)

Adding Higher-Order Model Terms

If you still have the driving data active in Design-Expert® software from Part 1 of this tutorial, continue on. If you exited the program, re-start it using our new opening screen (click Open Design) or use File, Open Design to open data file Drive time.dxpx. Otherwise, go back and set it up as instructed in One-Factor RSM Tutorial (Part 1 – The Basics). The wavy curve you see on the response surface plot for drive time is characteristic of a third-order (cubic) polynomial model. Could an even higher-order model be applied to the data from this case? If so, would it improve the fit? Under the Design branch click the Evaluation node.

Design evaluation

Change the Order to Quartic or double-click the term A4 to put it in the model (“M”).

Model changed to quartic (4th order)

Click Results to see the evaluation of this higher-order model.



Evaluation finds no aliases for quartic model

No aliases are found, but other aspects of the evaluation fall short of the ideal. Scroll down the output (or use the Bookmarks) and pay close attention to the annotations. On the floating Bookmarks click the button for Leverage.

Note the design point with the unusually high leverage of 0.9743. This is the late departure time near 50 minutes that occurred due to Mark oversleeping, causing a ‘botched’ factor setting. It should not be surprising to see this stand out so poorly for leverage.

Explore more advanced design evaluation statistics: Many more evaluation statistics can be generated from Design-

Expert – the ones shown by default are the most important ones. To enable additional measures and modify defaults,

click Options under the Model screen.

Press ahead to the Graphs to see the plot of FDS – fraction of design space. Click the curve of standard error at a fraction near 0.8 (80 percent) to generate cross-reference lines like those shown in the screen shot below.


FDS graph

Explore FDS graph: As noted in Screen Tips (hint: press the light-bulb icon), this is a line graph showing the

relationship between the “volume” of the design space (area of interest) and amount of prediction error. The curve

indicates what fraction (percentage) of the design space has a given prediction error or lower. In general, a lower and

flatter FDS curve is better. The FDS graph provides very helpful information on scaled prediction variance (SPV) for

comparing alternative test matrices – simple enough that even non-statisticians can see differences at a glance, and

versatile for any type of experiment – mixture, process, or combined. For example, one could rerun the FDS graph for

the cubic model and compare results and/or try some other experiment designs.

Let’s not belabor the evaluation: Go back to the Analysis branch and click the Drive time node. Then press ahead to the Model and change Process order to Quartic.

Changing model to quartic for analysis

Now click the ANOVA tab. Notice that not only does the A4 term come out insignificant (p-value of 0.91), but the Pred R-Squared goes negative – not a good sign!



ANOVA for quartic model (annotations turned of f via View menu)

Before moving on to the next topic, return to the Model tab and re-set the Process order to Cubic, which we recommend for this case.

Back to the cubic model

By the way, Design-Expert distinguishes enough in this simplistic one-factor case to add up to sixth-order terms to the model list. However, in some cases, you may need to use the Add Term entry field. For example, in a two-factor RSM you can add terms such as A2B4 or A3B2.

Propagation of Error (POE)

Seeing such a rapid increase in drive time predicted for late departures makes Mark more aware of how much the response depends on what time he leaves home. He realizes that a 5-minute deviation one way or the other would not be an unreasonable expectation. How will this cause the drive time to vary? Perhaps by aiming for a specific departure time, Mark might reduce drive-time variation caused by day-to-day differences when he leaves for work. Via its capability to calculate and plot propagation of error (POE), Design-Expert can provide enlightenment on these issues.

Click the Design branch to bring up the run-sheet (recipe procedure) for the experiment. Then right-click the column-header for Factor 1 (A:Departure) and select Edit Info.


Editing info for the input factor

For Std Dev enter 5.

Entering standard deviation for factor

Press OK and go back to the Analysis branch, click the Drive time node and go to Model Graphs.

Then from the View menu select Propagation of Error.

Plot for POE

Notice that POE is minimized at two times for departure, which correspond with flats on the wavy response plot you looked at earlier.



Explore how Design-Expert accounts for factor deviation: As you may have noticed by the legend on the model

graph, Design-Expert makes use of the knowledge on standard deviation of the factor(s) to adjust the confidence

intervals.

Variation in factor level now accounted for

For details on how this is done, contact Stat-Ease statisticians via [email protected].

Multiple Response Optimization

Ideally, Mark would like to leave as late as possible (to get more sleep every morning!) while minimizing his drive time – but making it the least variable. These goals can be established in Design-Expert software so it can look for the most desirable outcomes.

Under the Optimization branch, choose the Numerical node. For Departure, which comes up by default, click Goal and select maximize.

Setting goal for departure

The program pictures this goal as an upward ramp (/) to indicate that the higher this variable goes the more desirable it becomes.

Desirability ramp for departure – later the better (maximize)

Next, click the response for Drive time. For its Goal select minimize.



Drive time minimized

Notice the ramp now goes downward (\) to show that for this variable, lesser is better, that is, more desirable.

Lastly, to reduce variation in drive time caused by deviation in departure, click POE (Drive Time) and set its Goal to minimize.

Minimizing POE

Explore options for numeric optimization: Before pressing ahead, click the Options button.

Options for numeric optimization



The settings here will affect the hill-climbing algorithm that Design-Expert uses to find the most desirable combination

of variables. For details, check Help. Click OK to accept the defaults.

Press the Solutions tab to see in “ramps” view what Design-Expert recommends for the most desirable departure. The program now chooses a departure time at random and climbs up the desirability response surface. It repeats this process over and over, but in this case, the same point (within a value “epsilon” for the duplicate solution filter – see Optimization Options above) is found every time – a departure around 33 minutes beyond the earliest start acceptable by Mark for his morning commute. (Your result may vary somewhat due to the random starting points of the hill-climbing algorithm.)

Ramps view of most desirable solution (your results may vary from this)

Now Mark knows when it’s best to leave for work while simultaneously maximizing the departure (and gaining more ‘shut-eye’), minimizing his drive time, and minimizing propagation of error. The only thing that could possibly go wrong would be if all the other commuters learn how to use RSM and make use of Design-Expert. Mark hopes that none of you who are reading this tutorial live in his suburban neighborhood and work downtown.

DX9-05-2-MultifactorRSM.docx Rev. 5/2/14

Design-Expert 9 User’s Guide Multifactor RSM Tutorial 1


(Part 1/3 – The Basics)

Response Surface Design and Analysis

This tutorial, the first of three in this series, shows how to use Design-Expert® software for response surface methodology (RSM). This class of designs is aimed at process optimization. A case study provides a real-life feel to the exercise.

If you are in a rush to get the gist on design and analysis of RSM, hop past all the sidebars. However, if/when you find the time, it will be well worth the effort to explore these “by-the-ways.”

Explore more fundamental features of the software: Due to the specific nature of this case study, a number of

features that could be helpful to you for RSM will not be implemented in this tutorial. Many of these features are used in

the General One Factor, RSM One Factor, or Two-Level Factorial tutorials. If you have not completed all these

tutorials, consider doing so before starting this one.

We will presume that you are knowledgeable of the statistical aspects of RSM. For a good primer on the subject, see RSM Simplified (Anderson and Whitcomb, Productivity, Inc., New York, 2005). You will find overviews on RSM and how it’s done via Design-Expert in the on-line Help system. To gain a working knowledge of RSM, we recommend you attend our Response Surface Methods for Process Optimization workshop. Call Stat-Ease or visit our website for a schedule at www.statease.com.

The case study in this tutorial involves production of a chemical. The two most important responses, designated by the letter “y”, are:

y1 - Conversion (% of reactants converted to product)

y2 - Activity.

The experimenter chose three process factors to study. Their names and levels are shown in the following table.

Factor Units Low Level (-1) High Level (+1)

A – Time minutes 40 50

B – Temperature degrees C 80 90

C – Catalyst percent 2 3

Factors for response surface study

You will study the chemical process using a standard RSM design called a central composite design (CCD). It’s well suited for fitting a quadratic surface, which usually works well for process optimization.

Explore the makeup of a CCD: The three-factor layout for this CCD is pictured below. It is composed of a core

factorial that forms a cube with sides that are two coded units in length (from -1 to +1 as noted in the table above). The

stars represent axial points. How far out from the cube these should go is a matter for much discussion between

statisticians. They designate this distance “alpha” – measured in terms of coded factor levels. As you will see, Design-

Expert offers a variety of options for alpha.


2 Multifactor RSM Tutorial Design-Expert 9 User’s Guide

Central Composite Design for three factors

Assume that the experiments will be conducted over a two-day period, in two blocks:

1. Twelve runs: composed of eight factorial points, plus four center points.

2. Eight runs: composed of six axial (star) points, plus two more center points.


Start the program by finding and double clicking the Design-Expert software icon.

Welcome screen

Press OK on the welcome screen. Next click the blank-sheet icon on the left of the toolbar and then pick the Response Surface folder tab to show the designs available for RSM.

Response surface design tab



The default selection is the Central Composite design, which is used in this case study.

Explore some other RSM design choices: To see alternative RSM designs for three factors, click at far left on Box-

Behnken (notice 17 runs near the screen bottom) and Miscellaneous designs, where you find the 3-Level Factorial

option (32 runs, including 5 center points). Now go back and re-select Central Composite design.

Click the down arrow in the Numeric Factors entry box and Select 3 as shown below.

Selecting three numeric factors

Explore options for CCD construction: Before entering factors and ranges, click Options near the bottom of the

CCD screen. Notice that it defaults to a Rotatable design with the axial (star) points set at 1.68179 coded units from the

center – a conventional choice for the CCD.

Default CCD option for alpha set so design is rotatable

Many options are statistical in nature, but one that produces less extreme factor ranges is the “Practical” value for alpha.

This is computed by taking the fourth root of the number of factors (in this case 3¼ or 1.31607). See RSM Simplified

Chapter 8 “Everything You Should Know About CCDs (but dare not ask!)” for details on this practical versus other

levels suggested for alpha in CCDs – the most popular of which may be the “Face Centered” (alpha equals one). Press

OK to accept the rotatable value.

Using the information provided in the table on page 1 of this tutorial (or on the screen capture below), type in the details for factor Name (A, B, C), Units, and Low and High levels.


Completed factor form

You’ve now specified the cubical portion of the CCD. As you did this, Design-Expert calculated the coded distance “alpha” for placement on the star points in the central composite design.

Explore an alternative way to lay out a CCD: Alternatively, by clicking the “entered factor ranges in terms of

alphas” option you can control how far out the runs will go for each of your factors.

Now return to the bottom of the central composite design form. Leave Type at its default value of Full (the other option is a “small” CCD, which we do not recommend unless you must reduce the number of runs to the bare minimum). You will need two blocks for this design, one for each day, so click the Blocks field and select 2.

Selecting the number of blocks

Notice the software displays how this CCD will be laid out in the two blocks – for example, 4 center points will go in one and 2 in the other. Click Continue to reach the second page of the “wizard” for building a response surface design. You now have the option of identifying Block Names. Enter Day 1 and Day 2 as shown below.

Block names

Press Continue to enter Responses. Select 2 from the pull down list. Now enter the response Name and Units for each response as shown below.

Completed response form



At any time in the design-building phase, you can return to the previous page by pressing the Back button. Then you can revise your selections. Press Finish to view the design layout (your run order may differ due to randomization).

Explore options for modifying the design: Design-Expert offers many ways to modify the design and how it’s laid

out on-screen. Preceding tutorials, especially Part 2 for General One Factor, delved into this in detail, so go back and

look this over if you haven’t already. Click the Tips (light bulb icon) button for a refresher.

Design layout (only partially shown, your run order may differ due to randomization)

Save the Data to a File

Now that you’ve invested some time into your design, it would be prudent to save your work. Click the File menu item and select Save As.

You can now specify your File name (we suggest tut-RSM) to Save as type “*.dxpx” in the Data folder for Design-Expert (or wherever you want to Save in).

Enter the Response Data – Create Simple Scatter Plots

Assume that the experiment is now completed. Obviously at this stage the responses must be entered into Design-Expert. We see no benefit to making you type all the numbers, particularly with the potential confusion due to differences in randomized run orders. Therefore, use the File, Open Design menu and select RSM.dxpx from the Design-Expert program Data directory. Click Open to load the data.

Let’s examine the data, which came in with the file you opened (no need to type it in!). Move your cursor to Std column header and right-click to bring up a menu from which to select Sort Ascending (this could also be done via the View menu).

Sorting by Standard (Std) Order

Now right-mouse click the Select column header and choose Space Point Type.


Displaying the Point Type

Notice the new column identifying points as “Factorial,” “Center” (for center point), and so on. Notice how the factorial points align only to the Day 1 block. Then in Day 2 the axial points are run. Center points are divided between the two blocks.

Unless you change the default setting for the Select option, do not expect the Type column to appear the next time you run Design-Expert. It is only on temporarily at this stage for your information.

Before focusing on modeling the response as a function of the factors varied in this RSM experiment, it will be good to assess the impact of the blocking via a simple scatter plot. Click the Graph Columns node branching from the design ‘root’ at the upper left of your screen. You should see a scatter plot with factor A:Time on the X-axis and the Conversion response on the Y-axis.

Explore the color-coded correlation grid: The correlation grid that pops up with the Graph Columns can be very

interesting. First off, observe that it exhibits red along the diagonal—indicating the complete (r=1) correlation of any

variable with itself (Run vs Run, etc). Block versus run (or, conversely, run vs block) is also highly correlated due to

this restriction in randomization (runs having to be done for day 1 before day 2). It is good to see so many white squares

because these indicate little or no correlation between factors, thus they can be estimated independently.

For now it is most useful to produce a plot showing the impact of blocks because this will be literally blocked out in the analysis. Therefore, on the floating Graph Columns tool click the button where Conversion intersects with Block as shown below.



Plotting the effect of Block on Conversion

The graph shows a slight correlation (0.152) of conversion with blocks (day by day). Whether this is something to be concerned about would be a matter of judgment by the experimenter. However it may in this case be such a slight difference that it merits no further discussion. Bear in mind that whatever the difference may be it will be filtered out mathematically so as not to bias the estimation of factor effects.

Graph Columns feature for design layout

Change the Y Axis to Activity to see how it’s affected by the day-to-day blocking (not much!).


Changing response (resulting graph not shown)

Finally, to see how the responses correlate, change the X Axis to Conversion.

Plotting one response versus the other (resulting graph not shown)

Feel free to make other scatter plots, but the ones done thus far will be most valuable so it will be best to move on from here.

Explore the color-by tool: Notice you can also color selected factors, including run (default). For example, choose

Color by Block to see which points were run in block 1 (black) and block 2 (red).



Graph columns with points colored by Block

However, do not get carried away with this, because it will be much more productive to do statistical analysis first –

before drawing any conclusions.

Analyze the Results

Now let’s start analyzing the responses numerically. Under the Analysis branch click the node labeled Conversion. A new set of tabs appears at the top of your screen. They are arranged from left to right in the order needed to complete the analysis. What could be simpler?

Begin analysis of Conversion

Design-Expert provides a full array of response transformations via the Transform option. Click Tips for details. For now, accept the default transformation selection of None.

Now click the Fit Summary tab. At this point Design-Expert fits linear, two-factor interaction (2FI), quadratic, and cubic polynomials to the response. At the top is the response identification, immediately followed below, in this case, by a warning: “The Cubic

Design-Expert® Software

Correlation: 0.224

Color points by

Block

1

2

50.0 60.0 70.0 80.0 90.0 100.0

50.0

55.0

60.0

65.0

70.0

Conversion (%)

Ac

tiv

ity


Model and higher are Aliased.” Do not be alarmed. By design, the central composite matrix provides too few unique design points to determine all the terms in the cubic model. It’s set up only for the quadratic model (or some subset).

Next you will see several extremely useful tables for model selection. (To move through the display, use the floating Bookmarks palette or the side/bottom scroll bars). Each table is discussed briefly via sidebars in this tutorial on RSM.

The table of “Sequential Model Sum of Squares” (technically “Type I”) shows how terms of increasing complexity contribute to the total model.

Explore the Sequential Model Sum of Squares table: The model hierarchy is described below:

“Linear vs Block”: the significance of adding the linear terms to the mean and blocks,

“2FI vs Linear”: the significance of adding the two factor interaction terms to the mean, block, and linear terms

already in the model,

“Quadratic vs 2FI”: the significance of adding the quadratic (squared) terms to the mean, block, linear, and two-

factor interaction terms already in the model,

“Cubic vs Quadratic”: the significance of the cubic terms beyond all other terms.



Sequential Model Sum of Squares

For each source of terms (linear, etc.), examine the probability (“Prob > F”) to see if it falls below 0.05 (or whatever statistical significance level you choose). So far, Design-Expert is indicating (via underline) the quadratic model looks best – these terms are significant, but adding the cubic order terms will not significantly improve the fit. (Even if they were significant, the cubic terms would be aliased, so they wouldn’t be useful for modeling purposes.) Use the handy Bookmarks tool to advance to the next table for Lack of Fit tests on the various model orders.


Summary Table: Lack of Fit Tests

The “Lack of Fit Tests” table compares residual error with “Pure Error” from replicated design points. If there is significant lack of fit, as shown by a low probability value (“Prob>F”), then be careful about using the model as a response predictor. In this case, the linear model definitely can be ruled out, because its Prob > F falls below 0.05. The quadratic model, identified earlier as the likely model, does not show significant lack of fit. Remember that the cubic model is aliased, so it should not be chosen.

Look over the last table in the Fit Summary report, which provides “Model Summary Statistics” for the ‘bottom line’ on comparing the options.

Summary Table: Model Summary Statistics

The quadratic model comes out best: It exhibits low standard deviation (“Std. Dev.”), high “R-Squared” values, and a low “PRESS.”

The program automatically underlines at least one “Suggested” model. Always confirm this suggestion by viewing these tables.

Explore more details on model selection: From the main menu select Tips, Screen Tips or simply press the light-

bulb icon ( ) for more information about the procedure for choosing model(s).

Design-Expert allows you to select a model for in-depth statistical study. Click the Model

tab at the top of the screen to see the terms in the model.



Model results

The program defaults to the “Suggested” model shown in the earlier Fit Summary table.

Explore more details on model selection: If you want, you can choose an alternative model from the Process Order

pull-down list. (Be sure to try this in the rare cases when Design-Expert suggests more than one model.)

The options for process order

At this stage you could make use of the Add Term feature. This is subject to limitations that you’d best ask about via

[email protected].

Also, you could now manually reduce the model by clicking off insignificant effects. For example, you will see in a

moment that several terms in this case are marginally significant at best. Design-Expert provides several automatic

reduction algorithms as alternatives to the “Manual” method: “Backward,” “Forward,” and “Stepwise.” Click the

Selection list box to see these. From more details, try Screen Tips and/or search Help.

Click the ANOVA tab to produce the analysis of variance for the selected model.



Statistics for selected model: ANOVA table

The ANOVA in this case confirms the adequacy of the quadratic model (the Model Prob>F is less than 0.05.) You can also see probability values for each individual term in the model. You may want to consider removing terms with probability values greater than 0.10. Use process knowledge to guide your decisions.

Next, press the Bookmarks button for R-Squared to see that Design-Expert presents various statistics to augment the ANOVA. The R-Squared statistics are very good — near to 1.

Post-ANOVA statistics

Press forward to Coefficients to bring the following details to your screen, including the mean effect-shift for each block, that is, the difference from Day 1 to Day 1 in the response.



Coefficients for the quadratic model

Press Equations to bring the next section to your screen — the predictive models in terms of coded versus actual factors (shown side-by-side below). Block terms are left out. These terms can be used to re-create the results of this experiment, but they cannot be used for modeling future responses.

Final equation: coded versus actual

You cannot edit any ANOVA outputs. However, you can copy and paste the data to your favorite Windows word processor or spreadsheet. Also, as detailed in the One-Factor RSM tutorial, Design-Expert provides a tool to export equations directly to Excel in a handy format that allows you to ‘plug and chug’, that is, enter whatever inputs you like to generate predicted response. This might be handy for clients who are phobic about statistics. ; )

Diagnose the Statistical Properties of the Model

The diagnostic details provided by Design-Expert can best be grasped by viewing plots available via the Diagnostics tab. The most important diagnostic — normal probability plot of the residuals — appears by default.


Normal probability plot of the residuals

Data points should be approximately linear. A non-linear pattern (such as an S-shaped curve) indicates non-normality in the error term, which may be corrected by a transformation. The only sign of any problems in this data may be the point at the far right. Click this on your screen to highlight it as shown above.

Find the floating Diagnostics Tool palette on your screen.

Explore details on how Design-Expert displays residuals: Notice that residuals are “externally studentized” unless

you change their form on the floating tool palette (not advised). This has been discussed in prior tutorials. To recap:

Externally calculating residuals increases the sensitivity for detecting outliers.

Studentized residuals counteract varying leverages due to design point locations. For example, center points carry

little weight in the fit and thus exhibit low leverage.

Now go to the Diagnostics Tool and click Resid. vs Run.

Residuals versus run



Now you can see that, although the highlighted run does differ more from its predicted value than any other, there is really no cause for alarm.

Explore details on how Design-Expert displays residuals: Each button on the palette represents a different

diagnostics graph. Check out the other graphs if you like. Press Screen Tips along the way to get helpful details and

suggestions on interpretation. In this case, none of the graphs really indicates anything that invalidates the model, so

press ahead.

Next press the Influence side for another set of diagnostics, including a report detailed case-by-case residual statistics. Press the Screen Tips light-bulb icon for details on the Cook’s Distance that now displays.

Cook’s Distance — the first of the Influence diagnostics

Exit (X) out of the Screen Tips and move on to Leverage. This is best explained by the previous tutorial on One-Factor RSM so go back to that if you did not already go through it. Then skip ahead to DFBETAS, which breaks down the changes in the model to each coefficient, which statisticians symbolize with the Greek letter β, hence the acronym DFBETAS — the difference in betas. For the Term click the down-list arrow () and select A as shown in the following screen shot.


DFBETAS for term A

You can evaluate ten model terms (including the intercept) for this quadratic predictive model (see sidebar below for help).

Explore DFBETAS: Click outside the Term field, reposition your mouse over the Term field and simply scroll your

mouse wheel to quickly move up and down the list. In a similar experiment to this one, where the chemist changed

catalyst, the DFBETAS plot for that factor exhibited an outlier for the one run where its level went below a minimal

level needed to initiate the reaction. Thus, this diagnostic proved to be very helpful in seeing where things went wrong

in the experiment.

Now move on to the Report to bring up detailed case-by-case diagnostic statistics, many which have already been shown graphically.

Diagnostics report



Observe that one value of DFFITS is flagged in red. As we discussed in the General One-Factor Tutorial, this statistic stands for difference in fits. It measures change in each predicted value that occurs when that response is deleted.

Explore a footnote on the diagnostics report: The note below the table (“Predicted values include block

corrections.”) alerts you that any shift from block 1 to block 2 will be included for purposes of residual diagnostics.

(Recall that block corrections did not appear in the predictive equations shown in the ANOVA report.)

Given that only one diagnostic is flagged, there may be no real cause for alarm. However, to get a better feel for this discrepant statistic, press the DFFITS button for the graph.

DFFITS versus run graph

Pay attention to the control limits — they are blue. This indicates less cause for concern than red-lined outliers, that is, points outside of the plus-or-minus 2 values for DFFITS are not that unusual. Anyways, assume for purposes of this tutorial that the experiments found nothing out of the ordinary for the one run that went slightly out for DFFITS.

Examine Model Graphs

The residuals diagnosis reveals no statistical problems, so now let’s generate response surface plots. Click the Model Graphs tab. The 2D contour plot of factors A versus B comes up by default in graduated color shading.


Response surface contour plot

Explore how to visualize design point in multifactor space: Note that Design-Expert displays any actual point

included in the design space shown. In this case you see a plot of conversion as a function of time and temperature at a

mid-level slice of catalyst. This slice includes six center points as indicated by the dot at the middle of the contour plot.

By replicating center points, you get a very good power of prediction at the middle of your experimental region.

The floating Factors Tool palette appears with the default plot. Move this floating tool as needed by clicking and dragging the top blue border. The tool controls which factor(s) are plotted on the graph.

Explore the Factors Tool features: The Gauges option near the top of the tool is the default. Each factor listed has

either an axis label, indicating that it is currently shown on the graph, or a red slider bar, which allows you to choose

specific settings for the factors that are not currently plotted. All red slider bars default to midpoint levels of those

factors not currently assigned to axes. You can change factor levels by dragging their red slider bars or by right clicking

factor names to make them active (they become highlighted) and then typing desired levels into the numeric space near

the bottom of the tool palette. Give this a try.

Click the C:Catalyst toolbar to see its value. Don’t worry if the red slider bar shifts a bit — we will instruct you how to re-set it in a moment.

Factors tool showing factor C highlighted and value displayed



Left-Click the red bar with your mouse and drag it to the right.

Slide bar for C pushed right to higher value

As indicated by the color key on the left, the surface becomes ‘hot’ at higher response levels, yellow in the ’80’s, and red above 90 for conversion.

Explore coordinates on the graph using a crosshairs tool: To enable a handy tool for reading coordinates off

contour plots, go to View, Show Crosshairs Window. Now move your mouse over the contour plot and notice that

Design-Expert generates the predicted response for specific factor values corresponding to that point. If you place the

crosshair over an actual point, for example – the one at the far upper left corner of the graph now on screen, you also see

that observed value (in this case: 66).

Prediction at coordinates of 40 and 90 where an actual run was performed

P.S. See what happens when you press the Full option for crosshairs.

Now press the Default button on the floating Factors Tool to place factor C back at its midpoint.


Explore Sheet View on Factors Tool: Switch to Sheet View on the Factors Tool by clicking the Sheet button.

Factors tool – Sheet view

In the columns labeled Axis and Value you can change the axes settings or type in specific values for factors. Give this a

try. Then return to the Gauges view and press the Default button.

P.S. At the bottom of the Factors Tool is a pull-down list from which you can also select the factors to plot. Only the

terms that are in the model are included in this list. At this point in the tutorial this should be set at AB. If you select a

single factor (such as A) the graph changes to a One-Factor Plot. Try this if you like, but notice how Design-Expert

warns if you plot a main effect that’s involved in an interaction.

Perturbation Plot

Wouldn’t it be handy to see all your factors on one response plot? You can do this with the perturbation plot, which provides silhouette views of the response surface. The real benefit of this plot is when selecting axes and constants in contour and 3D plots. See it by mousing to the Graphs Tool and pressing Perturbation or pull it up via View from the main menu.

The Perturbation plot with factor A clicked to highlight it

For response surface designs, the perturbation plot shows how the response changes as each factor moves from the chosen reference point, with all other factors held constant at the reference value. Design-Expert sets the reference point default at the middle of the design space (the coded zero level of each factor).



Click the curve for factor A to see it better. The software highlights it in a different color as shown above. It also highlights the legend. (You can click it also – it is interactive!) In this case, at the center point, you see that factor A (time) produces a relatively small effect as it changes from the reference point. Therefore, because you can only plot contours for two factors at a time, it makes sense to choose B and C – and slice on A.

Contour Plot: Revisited

Let’s look at the plot of factors B and C. Start by clicking Contour on the floating Graphs tool. Then in the Factors Tool right click the Catalyst bar and select X1 axis by left clicking it.

Making factor C the x1-axis

You now see a catalyst versus temperature plot of conversion, with time held as a constant at its midpoint.

Contour plot of B:temperature versus C:catalyst

Design-Expert contour plots are highly interactive. For example, right-click up in the hot spot at the upper middle and select Add Flag.


Adding a flag

That’s enough on the contour plot for now — hold off until Part 3 of this tutorial to learn other tips and tricks on making this graph and others more presentable. Right click and Delete flag to clean the slate.

Deleting the flag

3D surface plot

Now to really get a feel for how the response varies as a function of the two factors chosen for display, select from the floating Graphs Tool the 3D Surface. You then will see three-dimensional display of the response surface. If the coordinates encompass actual design points, these will be displayed. On the Factors Tool move the slide bar for A:time to the right. This presents a very compelling picture of how the response can be maximized. Right click at the peak to set a flag.

3D response surface plot with A:time at high level

You can see points below the surface by rotating the plot. Move your mouse over the graph. When the cursor turns into a hand click and hold the left mouse-button and then drag.



Seeing a point beneath the surface

See an actual result predicted so closely lends credence to the model. Things are really looking up at this point!

Explore another way to rotate the 3D graph: Another way to do this is to select View, Show Rotation to bring up a

handy tool for this.

Rotation tool

Move your cursor over the tool. The pointer changes to a hand. Now use the hand to rotate the vertical or horizontal

wheel. Whether you use the rotation tool or simply grab the plot with your mouse, watch the 3D surface change. It’s

fun! What’s really neat is how it becomes transparent so you can see hidden points falling below the surface. Notice

how the points below the surface are shown with a lighter shade. The Stat-Ease program developers thought of

everything! Before moving on from here, go back to the Rotation tool and press Default to put the graph back in its

original angle. Notice that you can also specify the horizontal (“h”) and vertical (“v”) coordinates.

Remember that you’re only looking at a ‘slice’ of factor A (time). Normally, you’d want to make additional plots with slices of A at the minus and plus one levels, but let’s keep moving — still lots to be done for making the most of this RSM experiment.

Analyze the Data for the Second Response

This step is a BIG one. Analyze the data for the second response, activity. Be sure you find the appropriate polynomial to fit the data, examine the residuals and plot the response surface. Hint: The correct model is linear.

Before you quit, do a File, Save to preserve your analysis. Design-Expert will save your models. To leave Design-Expert, use the File, Exit menu selection. The program will warn you to save again if you’ve modified any files.



(Part 2/3 – Optimization)

Introduction

This tutorial shows how to use Design-Expert software for optimization experiments. It's based on the data from Multifactor RSM Tutorial Part 1. You should go back to that tutorial if you've not completed it.

For details on optimization, see our on-line program help. Also, Stat-Ease provides in-depth training in our Response Surface Methods for Process Optimization workshop. Call or visit our web site for information on content and schedules.

In this section, you will work with predictive models for two responses, yield and activity, as a function of three factors: time, temperature, and catalyst. These models are based on results from a central composite design (CCD) on a chemical reaction.

Click the open design icon (see below) and load the case study data modeled by Stat-Ease and saved to a file named RSM-a.dxpx.

Open design icon

To see a description of the file contents, click the Summary node under the Design branch at the left of your screen. Within the design status screen you can see we modeled conversion with a quadratic model and activity with a linear model, as shown below.

Design summary

Explore making tables easier to read on screen: Drag the left border and open the window to better see the

branch/node menu. You can also re-size columns with your mouse.

Click on the Coefficients Table node at the bottom branch.



Coefficients Table

This table provides a convenient comparison of the coefficients for all of the responses.

Explore how to compare models: Because the Coefficients Table is laid in terms of coded factors you can make

inferences about the relative effects. For instance, notice that the coefficient for AC (11.375) in the conversion equation

is much higher than the coefficients for Factor B (4.04057). This shows, for the region studied, that the AC interaction

influences conversion more than Factor B. The coefficients in the table are color-coded by p-value, making it easy to see

each term’s significance at a glance. In our example, we chose to use the full quadratic model. Therefore, some less

significant terms (shown in black) are retained, even though they are not significant at the 0.10 level.

P.S. Right click any cell to export this report to PowerPoint or Word for your presentation or report. Check it out: This

is very handy!

Numerical Optimization

Design-Expert software’s numerical optimization will maximize, minimize, or target:

A single response

A single response, subject to upper and/or lower boundaries on other responses

Combinations of two or more responses.

Under the Optimization branch to the left of the screen, click the Numerical node to start.

Setting numerical optimization criteria


Setting the Optimization Criteria

Design-Expert allows you to set criteria for all variables, including factors and propagation of error (POE). (We will detail POE later.) The program restricts factor ranges to factorial levels (plus one to minus one in coded values) — the region for which this experimental design provides the most precise predictions. Response limits default to observed extremes. In this case, you should leave the settings for time, temperature, and catalyst factors alone, but you will need to make some changes to the response criteria.

Now you get to the crucial phase of numerical optimization: assigning “Optimization Parameters.” The program uses five possibilities as a “Goal” to construct desirability indices (di):

Maximize, Minimize, Target->, In range, Equal to -> (factors only).

Desirabilities range from zero to one for any given response. The program combines individual desirabilities into a single number and then searches for the greatest overall desirability. A value of one represents the ideal case. A zero indicates that one or more responses fall outside desirable limits. Design-Expert uses an optimization method developed by Derringer and Suich, described by Myers, Montgomery and Anderson-Cook in Response Surface Methodology, 3rd edition, John Wiley and Sons, New York, 2009.

For this tutorial case study, assume you need to increase conversion. Click Conversion and set its Goal to maximize. As shown below, set Lower Limit to 80 (the lowest acceptable value, and Upper Limit to 100, the theoretical high.

Conversion criteria settings

You must provide both these thresholds so the desirability equation works properly. By default, thresholds will be set at the observed response range, in this case 51 to 97. By increasing the upper end for desirability to 100, we put in a ‘stretch’ for the maximization goal. Otherwise we may come up short of the potential optimum.

Now click the second response, Activity. Set its Goal to target-> of 63. Enter Lower

Limits and Upper Limits of 60 and 66, respectively. These limits indicate that it is most



desirable to achieve the targeted value of 63, but values in the range of 60-66 are acceptable. Values outside that range are not acceptable.

Activity criteria settings

The above settings create the following desirability functions:

1. Conversion:

if less than 80%, desirability (di) equals zero

from 80 to 100%, di ramps up from zero to one

if over 100%, di equals one

2. Activity:

if less than 60, di equals zero

from 60 to 63, di ramps up from zero to one

from 63 to 66, di ramps back down to zero

if greater than 66, di equals zero

Explore details on features available for numerical optimization: Recall that at your fingertips you’ll find advice for

using sophisticated Design-Expert software features by pressing the button to see Screen Tips on Numerical

Optimization. Close out Screen Tips by pressing X at the upper-right corner of its screen.

Changing Desirability Weights and the (Relative) Importance of Variables

You can select additional parameters called “weights” for each response. Weights give added emphasis to upper or lower bounds or emphasize target values. With a weight of 1, di varies from 0 to 1 in linear fashion. Weights greater than 1 (maximum weight is 10) give more emphasis to goals. Weights less than 1 (minimum weight is 0.1) give less emphasis to goals.

Explore changing weights: Weights can be quickly changed by ‘grabbing’ (clicking and dragging) the handles

(squares ▫) on desirability ramps. Try pulling the square on the left down and the square on the right up as shown

below.


Weights change by grabbing handles with mouse

This might reflect a situation where your customer says they want the targeted value (63), but if it must be missed due to

a trade-off necessary for other specifications, it would be better to err to the high side. Before moving on from here, re-

enter the Lower and Upper Weights at their default values of 1 and 1; respectively. This straightens them to their

original ‘tent’ shape ().

“Importance” is a tool for changing relative priorities to achieve goals you establish for some or all variables. If you want to emphasize one over the rest, set its importance higher. Design-Expert offers five levels of importance ranging from 1 plus (+) to 5 plus (+++++). For this study, leave the Importance field at +++, a medium setting. By leaving all importance criteria at their defaults, no goals are favored over others.

Explore details on desirability functions: For statistical details about how desirability functions are constructed – and

formulas for weights and importance – select Help, Topic Help in the main menu. Then click Contents. The

Optimization branch is intuitively already expanded for you, so choose Numerical Optimization “Statistical Details”

then Importance as shown on the screen-shot below. From here you can open various topics and look for any details

you need.

Details about optimization criterion “importance” found in program Help

When you are done viewing Help, close it by pressing X at the upper-right corner of the screen.

Now click the Options button to see what you can control for the numerical optimization.



Optimization Options dialog box

Explore options for numerical optimization: Press Help to get details on these options. One that you should

experiment with is the Duplicate Solution Filter, which establishes the “epsilon” (minimum difference) for eliminating

essentially identical solutions. After doing your first search for the optimum, go back to this Option and slide it one way

and the other. Observe what happens to the solutions presented by Design-Expert. If you move the Filter bar to the

right, you decrease the number. Conversely, moving the bar to the left increases the solutions.

Click OK to close Optimization Options.

Running the optimization

Start the optimization by clicking the Solutions tab. It defaults to the Ramps view so you get a good visual on the best factor settings and the desirability of the predicted responses.

Numerical Optimization Ramps view for Solutions (Your results may differ)

The program randomly picks a set of conditions from which to start its search for desirable results – your results may differ. Multiple cycles improve the odds of finding multiple local optimums, some of which are higher in desirability than others. Design-Expert then sorts


the results from most desirable to least. Due to random starting conditions, your results are likely to be slightly different from those in the report above.

Explore various solutions via Ramps view: The ramp display combines individual graphs for easier interpretation.

The colored dot on each ramp reflects the factor setting or response prediction for that solution. The height of the dot

shows how desirable it is. Press the different solution buttons (1, 2, 3,…) and watch the dots. They may move only very

slightly from one solution to the next. However, if you look closely at temperature, you should find two distinct

optimums, the first few near 90 degrees; further down the solution list, others near 80 degrees. (You may see slight

differences in results due to variations in approach from different random starting points.) For example, click the last

solution on your screen. Does it look something like the one below?

Second optimum at lower temperature, but conversion drops, so it is inferior

If your search also uncovered this local optimum, note that conversion falls off, thus making it less desirable than the

higher-temperature option.

The Solutions Tool provides three views of the same optimization. (Drag the tool to a convenient location on the screen.) Click the Solutions Tool view option Report.



Report on numerical optimization

Now select the Bar Graph view from the floating Solutions Tool.

Solution to multiple response optimization — desirability bar graph

The bar graph shows how well each variable satisfies the criteria: values near one are good.

Optimization Graphs

Press Graphs near the top of your screen to view a contour graph of overall desirability. On the Factors Tool palette, right-click C:Catalyst. Make it the X2 axis. Temperature then becomes a constant factor at 90 degrees.

1

1

1

0.565864

1

0.752239

Desirability

0.000 0.250 0.500 0.750 1.000

A:time

B:temperature

C:catalyst

Conversion

Activity

Combined


Desirability graph (after changing X2 axis to factor C)

The screen shot above is a graph displaying graduated colors – cool blue for lower desirability and warm yellow for higher.

Design-Expert software sets a flag at the optimal point. To view the responses associated with the desirability, select the desired Response from its droplist. Take a look at the Conversion plot.

Conversion contour plot (with optimum flagged)



Explore Graph Preferences for contour graphs: Right click over this graph and choose Graph Preferences. Then

go to Surface Graphs and click Show contour grid lines.

Show contour grid lines option

Grid lines help locate the optimum, but for a more precise locator right-click the flag and Toggle Size to see the

coordinates plus many more predicted outcome details. To get just what you want on the flag, right-click it again and

select Edit Info.

Flag size toggled to see select detail

By returning to Toggle size, you can change back to the smaller flag. If you like, view optimal activity response as well.

To look at the desirability surface in three dimensions, again click Response and choose Desirability. Then, on the floating Graphs Tool, press 3D Surface. Next select View, Show Rotation and change horizontal control h to 170. Press your Tab key or click the graph. What a spectacular view!


3D desirability plot

Now you can see there’s a ridge where desirability can be maintained at a high level over a range of catalyst levels. In other words, the solution is relatively robust to factor C.

Explore Graph Preferences for 3D graphs: Right-click over your graph to re-summon Graph preferences. Via the

Surface Graphs tab change the 3D graph shading option to Wire Frame.

3D graph shading changed to wire frame

One way or another, please show your colleagues what Design-Expert software does for pointing out the most desirable

process factor combinations. We’d like that! The best way to show what you’ve accomplished is not on paper, but

rather by demonstrating it on your computer screen or by projecting your output to larger audiences. In this case, you’d

best shift back to the default colors and other display schemes. Do this by pressing the Default button Surface Graphs

and any other Graph Preference screens you experimented on.

P.S. Design-Expert offers a very high Graph resolution option. Try this if you like, but you may find that the processing

time taken to render this, particularly while rotating the 3D graph, can be a bit bothersome. This, of course, depends on

the speed of your computer and the graphics-card capability.

Now move on to graphical optimization—this may the best way to convey the outcome of an RSM experiment by displaying the “sweet spot” for process optimization.



Graphical Optimization

When you generated numerical optimization, you found an area of satisfactory solutions at a temperature of 90 degrees. To see a broader operating window, click the Graphical node. The requirements are essentially the same as in numerical optimization:

80 < Conversion

60 < Activity < 66

For the first response – Conversion (if not already entered), type in 80 for the Lower Limit. You need not enter a high limit for graphical optimization to function properly.

Graphical optimization: Conversion criteria

Click Activity response. If not already entered, type in 60 for the Lower Limit and 66 for the Upper Limit.

Now click the Graphs button to produce the “overlay” plot. Notice that regions not meeting your specifications are shaded out, leaving (hopefully!) an operating window or “sweet spot.” Now go to the Factors Tool palette and right-click C:Catalyst. Make it the X2 axis. Temperature then becomes a constant factor at 90 degrees as before for Solution 1.


Overlay plot

Notice the flag remains planted at the optimum. That’s handy! This Design-Expert display may not look as fancy as 3D desirability but it can be very useful to show windows of operability where requirements simultaneously meet critical properties. Shaded areas on the graphical optimization plot do not meet the selection criteria. The clear “window” shows where you can set factors that satisfy requirements for both responses.

Explore adding uncertainty intervals around your window of operability: Go back to the Criteria and click Show

Interval (one-sided) for both Conversion and Activity. This provides a measure of uncertainty on the boundaries

predicted by the models — a buffer of sorts.

Confidence intervals (CI) superimposed on operating window

After looking at this, go back and turn off the intervals to re-set the graph to the default settings.

P.S. If you are subject to FDA regulation and participate in their quality by design (QBD) initiative, the CI-bounded

window can be considered to be a functional design space, that is, a safe operating region for any particular unit

operations. However, to establish a manufacturing design space one must impose tolerance intervals. This tutorial

experiment provided too few runs to support imposition of TIs. To size designs properly for manufacturing QBD

requires advanced know-how taught by Stat-Ease in its Designed Experiments for Pharma workshop.



Let’s say someone wonders whether the 80 minimum for conversion can be increased. What will this do to the operation window? Find out by dragging the 80 conversion contour until it reaches a value near 90. Then right-click it and Set contour value to 90 on the nose.

Changing the conversion specification to 90 minimum

It appears that the more ambitious goal of 90 percent conversion is feasible. This requirement change would make the lower activity specification superfluous as evidenced by it no longer being a limiting level, that is, not a boundary condition on the operating window.

Graphical optimization works great for two factors, but as factors increase, optimization becomes more and more tedious. You will find solutions much more quickly by using the numerical optimization feature. Then return to the graphical optimization and produce outputs for presentation purposes.

Response Prediction and Confirmation

This feature in Design-Expert software allows you to generate predicted response(s) for any set of factors. To see how this works, click on the Point Prediction node (lower left on your screen).

Click the Point Prediction node (left on your screen). Notice it now defaults to the first solution. (Be thankful Design-Expert programmers thought of this, because it saves you the trouble of dialing it up on the Factors Tool.)

Point prediction set to solution #1


Explore other factor settings by moving the slide bar: The Factors Tool again allows you to adjust the settings to any

values you wish. Go ahead and play with them now if you like. You can either move the slider controls, or switch to the

Sheet view and enter values.

P.S. Take a moment now to study the screen tips on all the statistical intervals that come up when you press the light-

bulb icon ( ).

Confirmation

After finding the optimum settings based on your RSM models, the next step is to confirm that they actually work. To do this, click the Confirmation node (left side of your screen).

Confirmation predictions set to solution #1

Look at the 95% prediction interval (“PI low” to “PI high”) for the Activity response. This tells you what to expect for an individual (n = 1) confirmation test on this product attribute. You might be surprised at the level of variability, but it will help you manage expectations. (Note: block effects, in this case day-by-day, cannot be accounted for in the prediction.)

Of course you would not convince many people by doing only one confirmations run. Doing several would be better. For example, let’s say that the experimenters do three confirmatory tests. Go to the Confirmation Tool and enter for n the number 3. Click the Enter Data option and type for Activity 62, 63 and 64.

Entering confirmation run results

Notice that the prediction interval (PI) narrows as n increases. Does the Data Mean (displayed in red) fall within this range? If so, the model is confirmed.



Explore increasing numbers of confirmation runs: Keep increasing the value for n. Observe the diminishing

returns in terms of the precision, that is, the PI approaches a limit — the confidence interval (CI) that you saw in Point

Prediction. The CI is a function of the number of experimental runs from which the model is derived. That is done is

this stage, so one can only go so far with the number of confirmation runs. Perhaps half a dozen of these may suffice.


Now that you’ve invested all this time into setting up the optimization for this design, it would be prudent to save your work. Click the File menu item and select Save As. You can now specify the File name (we suggest tut-RSM-opt) for Save as type “*.dxpx” in the Data folder for Design-Expert (or wherever you want to Save in).

If you are not worn out yet, you will need this file in Part 3 of this series of tutorials.

Summary

Numerical optimization becomes essential when you investigate many factors with many responses. It provides powerful insights when combined with graphical analysis. However, subject-matter knowledge is essential to success. For example, a naive user may define impossible optimization criteria that results in zero desirability everywhere! To avoid this, try setting broad acceptable ranges. Narrow them down as you gain knowledge about how changing factor levels affect the responses. Often, you will need to make more than one pass to find the “best” factor levels that satisfy constraints on several responses simultaneously.

This tutorial completes the basic introduction to doing RSM with Design-Expert software. Move on to the next tutorial on advanced topics for more detailing of what the software can do. If you want to learn more about response surface methods (not the software per se), attend our Stat-Ease workshop Response Surface Methods for Process Optimization.

We appreciate your questions and comments on Design-Expert software. E-mail these to [email protected] along with your dx* file if you’ve created one. Do so at the design stage to get advice on a planned experiment and/or when you’ve collected the data and taken a shot at analyzing it. It’s always good to get a second opinion from experts in the field. Don’t be shy!




(Part 3/3 – Advanced Topics)

Tips and Tricks for Making Response Graphs Most Presentable

Go back a step and re-open the file named RSM-a.dxpx. Then under the Analysis branch click the R1:Conversion node and go to Model Graphs to bring up the contour plot. Let’s quickly try some things here that you may find useful when making a presentation.

In the vacant region of the AB contour plot right click and select Add contour. Then drag the contour around (it will become highlighted). You may get two contours from one click like those with the same response value shown below. (This pattern indicates a shallow valley, which becomes apparent when we get to the 3D view later.)

Adding a contour

Click the new contour line to highlight it. Then drag it (place the mouse cursor on the contour and hold down the left button while moving the mouse) to as near to 81 as you can. Now to obtain the precise contour level, right-click the contour you just dragged, choose Set contour value and enter 81.

Setting a contour value

Explore another way to set contour values: Right click over the plot and choose Graph Preferences. Then choose

Contours. Now select the Incremental option and fill in Start at 66, Step at 3, and Levels at 8.



Setting contour values incrementally via Graph Preferences

If you go this route, be sure to look over the Min and Max values first. That gives you a clue on where to start and how

big to step on the contour values.

To zoom in on the area around the center point (the red dot labeled “6”), position the crosshairs and, while holding down the left mouse button, drag over (rope-off) your desired region of interest.

Zooming in on a region of interest by roping off a box

Notice how the graph coordinates change. Obviously you would now want to add more contours using the tools you learned earlier in this tutorial. However, do not spend time on this now: Right-click over the graph and select Default View Window.

Restoring default region (factorial ranges within CCD)

That’s enough for the contours plot for now. On the Graphs Tool go to 3D Surface view. Modify the color range via a right-click over the gradient, which brings up the Edit Legend


dialog box. Change the Low to 80 and the High to 90. Notice how this makes the graph far more colorful and thus informative on the relative heights.

Edit Legend dialog box to change the color gradient

Now click the design point sticking up in the middle. See how this is identified in the legend at the left by run number and conditions.

3D graph enhanced for color gradient with point click and identified

Now try a handy feature for pulling up the right plot for any given run. On the Factors Tool select off the Run # down-list number 1. The 3D view now shifts to the correct ‘slice’ on factor C (catalyst). However the colors are not ideal now. So right-click over the gradient and in the Edit Legend dialog box press the Defaults button. Your graph should now match the one shown below.



Jump to run feature demonstrated

By the way, if you put in any comments on a particular run, it will show in this view with the point having been selected.

Explore adding a comment: Hearken back to the General One-Factor tutorial where we demonstrated how to add a

comment using tools in the Design layout. This can be very useful to document unusual happenings during any given

run.

Much more can be done for your show-and-tell. Spend time beforehand to try different things that Design-Expert can do. Take advantage of default buttons to put things back the way they were.

Adding Propagation of Error (POE) to the Optimization

Details about the variation in your input factors can be fed into Design-Expert® software. Then you can generate propagation of error (POE) plots showing how that error is transmitted to the response. Look for conditions that minimize transmitted variation, thus creating a process that’s robust to factor settings. This tutorial shows how to generate POE from an experiment designed by response surface methods (RSM).

Explore simpler case for application of POE: Propagation of error is covered in the One Factor RSM tutorial in a

way that is far easier to see, so be sure to review this if you want to develop a fuller understanding of this mathematical

tool.

To be sure we start from the same stage of analysis, re-open the file named RSM-a.dxpx. Then click the Design node on the left side of the screen to get back to the design layout.

Next select View, Column Info Sheet. Enter the following information into the Std. Dev.

column: time: 0.5, temperature: 1.0, catalyst: 0.05, as shown below.


Column Info Sheet with factor standard deviations filled in

Notice the software already entered the standard deviation for the analyzed response, Conversion (4.1…). Because you haven’t changed any other data, Design-Expert remembers your previous analysis choices — you can simply click through the analysis buttons.

Explore another way to specify variation in response: For your information, right-clicking the buttons to the left of

response names allows you to specify a different standard deviation.

Option to enter a different standard deviation for response

Otherwise the field will be protected, that is, you cannot alter it.

Under the Analysis branch click the Conversion node. Then jump past the

intermediate buttons for analysis and click the Model Graphs tab. Select View,

Propagation of Error. (This option was previously grayed out — unavailable — because the standard deviations for the factors had not yet been entered.)

Contour graph for POE

Now on the floating Graphs Tool select 3D Surface.



3D Surface view of the POE Graph

The lower the POE the better, because less of the error in control factors will be transmitted to the selected response, which results in a more robust process.

Explore what happens if you try to generate POE on a response that’s fitted to a linear model: When the surface is

linear, such as that for Activity, error is transmitted equally throughout the region. Therefore, Design-Expert software

‘grays out’ the option for propagation of error. See this for yourself by trying it.

Now that you’ve generated POE for Conversion, let’s go back and add it to the optimization criteria. Under the Optimization branch click the Numerical node. For the POE (Conversion) set the Goal to minimize with a Lower Limit of 4 and an Upper Limit

of 5 as shown below.

Set Goal and Limits for POE (Conversion)


Now click the Solutions button atop the screen to generate new solutions with additional

criteria. On the floating Solutions Tool click Ramps. (Note: Due to random starting points for the searches, you may see slight differences on your screen versus the shot below.)

Ramps view for optimization with POE (Your results may differ)

The above optimal solution represents the formulation that best maximizes conversion and achieves a target value of 63 for activity, while at the same time finds the spot with the minimum error transmitted to the responses. So, this should represent process conditions that are robust to slight variations in factor settings. In this case it does not make much of a difference whether POE is accounted for or not (go back and check this out for yourself). However, in some situations it may matter, so do not overlook the angle of POE.

Design Evaluation

Design-Expert offers powerful tools to evaluate RSM designs. Design evaluation ought to be accomplished prior to collecting response data, but it can be done after the fact. For example, you may find it necessary to change some factor levels to reflect significant deviations from the planned set point. Or you may miss runs entirely – at least for some responses. Then it would be well worthwhile to re-evaluate your design to see the damage.

For a re-cap of what’s been done so far, go to the Design branch and click the Summary node.

Design summary

The summary reports that the experimenter planned a central composite design (CCD) in two blocks, which was geared to fit a quadratic model. Click the Evaluation node and notice Design-Expert assumes you want details on this designed-for order of model.



Design evaluation – model choice

Click the Results tab for an initial report showing annotations on by default.

Design evaluation results

Scroll through the results or use the handy Bookmarks palette and note the results look very good – as you’d expect from a standard design for RSM.

Explore a really bad factor matrix: For a design that produces a far worse evaluation, take a look at the Historical

Data RSM Tutorial.

Press ahead to the Graphs tab atop the screen. It defaults to the FDS Graph that depicts standard error versus the fraction of design space. Click the curve you see depicted. Design-Expert now provides coordinate lines for easy reading.


FDS (fraction of design space) graph with coordinates clicked on

Based on extensive sampling of the experimental region (50,000 points by default as noted in the legend), the “y” axis on the FDS graph quantifies the maximum prediction variability at any given fraction of the total space. For example, as noted in the legend at the left of the screen, 80 percent of this response surface method (RSM) design falls at or below ~0.5 units of standard error (SE). Due to the random sampling algorithm, your FDS may vary a bit. When you evaluate alternative designs, favor those with lower and flatter FDS curves.

Explore FDS for sizing designs: FDS provides the mechanism for sizing RSM designs based on how precisely you

need to predict your response. Stat-Ease teaches how to do this in its workshop on RSM. For statistical details, see the

talk on “Practical Aspects of Algorithmic Design of Physical Experiments” posted to the webinar page at our website, or

click http://www.statease.com/webinars/practical_aspects_of_algorithmic_design_of_physical_experiments.pdf.

The FDS provides insights on prediction capabilities. To view design ‘rotatability’ criteria, select View, Contour. Design-Expert then displays the standard error plot, which shows how variance associated with prediction changes over your design space.

http://www.statease.com/webinars/practical_aspects_of_algorithmic_design_of_physical_experiments.pdf



Standard error contour plot

You can see the central composite design (CCD) provides relatively precise predictions over a broad area around the 6 center points. Also, notice the circular contours. This indicates the desirable property of rotatability – equally precise predictive power at equal distances from the center point of this RSM design. For standard error plots, Design-Expert defaults to black and white shading. The graduated shading that makes normal response contour plots so colorful will not work when displaying standard error. Look closely at the corners of this graph and notice they are gray, thus indicating regions where the response cannot be predicted as precisely.

Explore the outer regions of a central composite design: See what happens when you extrapolate beyond

experimentation regions by following these steps. First select Display Options, Process Factors, Coded. Then right-

click over the graph and select Graph Preferences. Change the default X1 Axis values for Low to -2 and High to 2.

Next, click the X2 Axis tab and change Low value to -2 and High value to 2. After completing these changes, press OK.

You now should see a plot like that shown below.

Contour plot of standard error with expanded axes, extrapolated area shaded

Design-Expert® SoftwareFactor Coding: CodedStd Error of Design

Design PointsStd Error Shading

1.500

0.500

X1 = A: timeX2 = B: temperature

Coded FactorC: catalyst = 0.000

-2.00 -1.20 -0.40 0.40 1.20 2.00

-2.00

-1.20

-0.40

0.40

1.20

2.00Std Error of Design

A: time (min.)

B: te

mp

era

ture

(d

eg

C)

0.5

1

1.5 1.5

1.5 1.5

6


As shown in the key, shading begins at one-half standard deviation and increases linearly up to 1.5 times standard

deviation. So long as you stay within specified factorial ranges (plus/minus 1), shading remains relatively light —

beyond that the plot darkens. Be wary of predictions in these nether regions! Before leaving this sidebar exploration, go

back to Graph Preferences and reset both axes to their defaults. Also, change factor back to their actual levels.

Now on the floating Graphs Tool click 3D Surface.

3D view of standard error

3D plot of standard error

Notice the flat bottom in this bowl-shaped surface of standard error. That’s very desirable for an RSM design. It doesn’t get any better than this!

DX9-05-3-HistRSM-V8.docx Rev. 5/2/14

Design-Expert 8 User’s Guide Historical Data RSM Tutorial – Part 1 1

Historical Data RSM Tutorial

Part 1 – The Basics

Introduction

In this tutorial you will see how the regression tool in Design-Expert® software, intended for response surface methods (RSM), is applied to historical data. We don’t recommend you work with such happenstance variables if there’s any possibility of performing a designed experiment. However, if you must, take advantage of how easy Design-Expert makes it to develop predictive models and graph responses, as you will see by doing this tutorial. It is assumed that at this stage you’ve mastered many program features by completing preceding tutorials. At the very least you ought to first do the one-factor RSM tutorials, both basic and advanced, prior to starting this one.

The historical data for this tutorial, shown below, comes from the U.S. Bureau of Labor Statistics via James Longley (An Appraisal of Least Squares Programs for the Electronic Computer from the Point of View of the User, Journal of the American Statistical Association, 62 (1967): 819-841). As discussed in RSM Simplified (Mark J. Anderson and Patrick J. Whitcomb, Productivity, Inc., New York, 2005: Chapter 2), it presents some interesting challenges for regression modeling.

Run A: Prices

B: GNP

C: Unemp.

D: Military

E: Pop.

F: Time

Employ.

# (1954 =100)

Armed Forces

People >14

Year Total

1 83 234289 2356 1590 107608 1947 60323

2 88.5 259426 2325 1456 108632 1948 61122

3 88.2 258054 3682 1616 109773 1949 60171

4 89.5 284599 3351 1650 110929 1950 61187

5 96.2 328975 2099 3099 112075 1951 63221

6 98.1 346999 1932 3594 113270 1952 63639

7 99 365385 1870 3547 115094 1953 64989

8 100 363112 3578 3350 116219 1954 63761

9 101.2 397469 2904 3048 117388 1955 66019

10 104.6 419180 2822 2857 118734 1956 67857

11 108.4 442769 2936 2798 120445 1957 68169

12 110.8 444546 4681 2637 121950 1958 66513

13 112.6 482704 3813 2552 123366 1959 68655

14 114.2 502601 3931 2514 125368 1960 69564

15 115.7 518173 4806 2572 127852 1961 69331

16 116.9 554894 4007 2827 130081 1962 70551

Longley data on U.S. economy from 1947-1962

2 Historical Data RSM Tutorial – Part 1 Design-Expert 8 User’s Guide

Assume the objective for analyzing this data is to predict future employment as a function of leading economic indicators – factors labeled A through F in the table above. Longley’s goal was different: He wanted to test regression software circa 1967 for round-off error due to highly correlated inputs. Will Design-Expert be up to the challenge? We will see!

Let’s begin by setting up this “experiment” (quotes added to emphasize this is not really an experiment but rather an after-the-fact analysis of happenstance data).

Design the “Experiment”

Click the Design-Expert icon that may appear on your desktop. You will see our handy new easy-start opening page. (To save you typing time, we will re-build a previously saved design rather than entering it from scratch.) Click Open Design as highlighted below.

New easy-start page – Open Design option highlighted in red

The file name is Longley.dxp. Double-click to open.

Opening the Longley data

The data table appears on your screen. To re-build this design (and thus see how it was created), press the blank-sheet icon at the left of the toolbar (or select File, New Design).

New Design icon

Click Yes when Design-Expert queries “Use previous design info?”



Re-using previous design

Now you see how this design was created via the Response Surface tab and Historical Data option.

Setting up historical data design

Note for each of 6 numeric factors we entered name, units, and range from minimum (“Min”) to maximum (“Max”). Before moving ahead, you must set Design-Expert to how many rows of data you want to key or copy/paste into the design layout. In this case there are 16 rows.

Entry for rows

Press Continue to accept all entries on your screen. You now see response details – in this case only one response.


Response entry

Press Continue to see the resulting design layout in run order. (Ignore the column labeled “Std” because there is no standard order for happenstance data).

A Peculiarity on Pasting Data

You could now type in all data for factor levels and resulting responses, row-by-row. (Don’t worry: We won’t make you do this!) However, in most cases data is already available via a Microsoft Window-based spreadsheet. If so, simply click/drag these data, copy to Window’s clipboard, and Edit, Paste (or right-click and Paste as shown below) into the design layout within Design-Expert. (Be sure, as shown below, to first click/drag the top row of all your destination cells.)

Correct way to paste data into Design-Expert (top-row of cells pre-selected)

If you simply click the upper left cell in the empty run sheet, the program only pastes one value.

Analyze the Results

Normally you’d save your work at this stage, but because we already did this, simply re-open our file: Press the Open Design icon and double-click Longley.dxp. Click No to pass up the opportunity to save what you did previously.

Last chance to save (say “No” in this case)

Before we get started, be forewarned you will encounter many statistics related to least squares regression and analysis of variance (ANOVA). If you are coming into



this without previous knowledge, pick up a copy of RSM Simplified and keep it handy. For a good guided tour of statistics for RSM analysis, attend our Stat-Ease workshop titled RSM for Process Optimization. Details about this computer-intensive, hands-on class – including prerequisites – are at www.statease.com.

Under the Analysis branch, click the Employment branch. Design-Expert displays a screen for transforming response. However, as noted by the program, the response range in this case is so small that there is little advantage to applying any transformation.

Information about the response shown on the Transformation screen

Press Fit Summary. Design-Expert evaluates each degree of the model from the mean on up. In this case, the best that can be done is linear. Anything higher is aliased.

Fit Summary – only the linear model is possible here

Move on by pressing Model.



Linear model is chosen

It’s all set up how Design-Expert suggested. Notice many two-factor interactions can’t be estimated due to aliasing – symbolized by a red tilde (~). Hold on to your hats (because this upcoming data is really a lot of hot air!) and press ANOVA (analysis of variance).

Analysis of variance (ANOVA)

Notice although the overall model is significant, some terms are not.



Some statistical details on how Design-Expert does analysis of variance You may have noticed this ANOVA is labeled “[Partial sum of squares - Type III]. This approach to ANOVA, done by default, causes total sums-of-squares (SS) for the terms to come up short of the overall model when analyzing data from a non-orthogonal array, such as historical data. If you want SS terms to add up to the model SS, go to Edit, Preferences for Math and change the default to Sequential (Type I). However, we do not recommend this approach because it favors the first term put into the model. For example, in this case, ANOVA by partial SS (Type III -- the default of DX) for the response (employment total) calculates prob>F p-value for A as 0.8631 (F=0.031) as seen above, which is not significant. Recalculating ANOVA by sequential sum of squares (Type I) changes the p to <0.0001 (F=1876), which looks highly significant, but only because this term (main effect of factor A) is fit first. This simply is not correct.

Assuming Factor A (population) is least significant of all as indicated by default ANOVA (partial SS), let’s see what happens with it removed. However, before we do, on the Bookmarks, click R-Squared and view statistics (shown below) to help us compare what happens before and after reducing the model.

Model statistics

Also bookmark the Coefficients estimates.

Coefficient estimates for linear model

Notice the huge VIF (variance inflation factor) values. A value of 1 is ideal (orthogonal), but a VIF below 10 is generally accepted. A VIF above 1000, such as factor B (GNP), indicates severe multicollinearity in the model coefficients. (That’s bad!). In the follow-up tutorial (Part 2) based on this same Longley data, we delve more into this and other statistics generated by Design-Expert for purposes of design evaluation. For now, right-click any VIF result to access context-sensitive Help, or go to Help on the main menu and search on this statistic. You will find some details there.

Press Model again. Right-click A-Prices and Exclude it, or simply double-click this term to remove the “M” (model) designation.


Excluding an insignificant term

You could now go back to ANOVA, look for the next least significant term, exclude it, and so on. However, this backward-elimination process can be performed automatically in Design-Expert. Here’s how. First, reset Process Order to Linear.

Resetting model to linear

Now change Selection to Backward.

Specifying backward regression

Notice a new field called “Alpha out” appears. By default the program removes the least significant term, step-by-step, as long as it exceeds the risk level (symbolized by statisticians with the Greek letter alpha) of 0.1 (estimated by p-value). Let’s be a bit more conservative by changing Alpha out to 0.05.



Changing risk-level alpha for removing model terms via backward selection

Now press ANOVA to see what happens.

Backward regression results

Not surprisingly, the program first removed A and then E – that’s it. Look at the following ANOVA table and note all other terms come out significant. (Note: If you do not see the report of the model being “significant” change your View to Annotated ANOVA.)

ANOVA for backward-reduced model

You may have noticed that in the full model, factor B had a much higher p-value than what’s shown above. This instability is typical of models based on historical data. Scroll down the ANOVA table to view model statistics and coefficients.


Backward-reduced model statistics and coefficients

Now let’s try a different regression approach – building the model from the ground (mean) up, rather than tearing terms down from the top (all terms in chosen polynomial). Press Model, then re-set Process Order to Linear. This time choose Forward stepwise regression in Selection. To provide a fair comparison of this forward approach with that done earlier going backward, change Alpha in to 0.05.

Forward selection (remember to re-set model to the original process order first!)

Heed the text displayed by the program (When reducing your model…) because this approach may not work as well for this highly collinear set of factors. See what happens now in ANOVA.

Results of forward regression



Surprisingly, factor B now comes in first as the single most significant factor. Then comes factor C. That’s it! The next most significant factor evidently does not achieve the alpha-in significance threshold of p<0.05.

Look at your ANOVA to see that all the other terms come out significant.

ANOVA for forward-reduced model

On Bookmarks, click R-Squared.

Forward-reduced model statistics and coefficients

This simpler model scores very high on all measures of R-squared, but it falls a bit short of what was achieved in the model derived from the backward regression.

Finally, go back to Model, re-set Process Order to Linear and try the last model Selection option offered by Design-Expert software: Stepwise.

Specifying stepwise regression


As you might infer from seeing both Alpha in and Alpha out now displayed, stepwise algorithms involve elements of forward selection with bits of backward added in for good measure. For details, search program Help, but consider this – terms that pass the alpha test in (via forward regression) may later (after further terms are added) become disposable according to the alpha test out (via backward selection). If this seems odd, look back at how factor B’s p-value changed depending on which other factors were chosen with it for modeling. To see what happens with this forward-selection method, press ANOVA again. Results depend on what you do with Alpha in and Alpha out – both which default back to 0.1000.

As you see in the message displayed for both forward and stepwise (in essence an enhancement of forward) approaches, we favor the backward approach if you decide to make use of an automated selection method. Ideally, an analyst is also a subject-matter expert, or such a person is readily accessible. Then they could do model reduction via the manual method filtered not only by the statistics, but also by simple common sense from someone with profound system knowledge.

This concludes part 1 of our Longley data-set exploration. In Part 2 we mine deeper into Design-Expert to see interesting residual analysis aspects within Diagnostics, and we also see what can be gleaned from its sophisticated tools within Design, Evaluation.

Part 2 – Advanced Topics

Design Evaluation

If you still have the Longley data active in Design-Expert software from Part 1 of this tutorial, continue on. If you exited the program, re-start it and use Open

Design to open your data file (Longley.dxp). Under the Design branch of the program, click Evaluation. The software brings up a quadratic polynomial model by default, but, as you will see, the order must be downgraded to linear (we will get to the reason momentarily). The screen shot shows the Response field set at “Design Only” as opposed to the Employment response. In other words, it will evaluate the entire matrix of factors, regardless whether response data are present. The other option (response by response) comes in handy when experimenters end up with missing data, thus degrading the “designed-for” model.



Design evaluation (design only)

Press the Results button.

Results of evaluation for quadratic polynomial


This model is badly aliased. For example, the effect of A is confounded with -24.5 CD, etc. Go back to Model and reduce the Order to Linear.

Re-setting order to linear

Press Results again and note “No aliases found…” Much better!

Results of evaluation for linear model

On Bookmarks click the DF option to bring up the accounting for degrees of freedom.

Bookmarking to evaluate degrees of freedom (DF)

Looking over the annotations provided by the software (activated via View, Annotated Evaluation), notice this design flunks the recommendation for pure error df. Of course this really is not a designed experiment, but rather historical data collected at happenstance.



Annotations for degrees of freedom

Study the next section of the evaluation by Design-Expert. Do any of the statistics pass the tests suggested for a good design? No!

Details on model terms, including power

Scroll down or bookmark to the leverage report. These statistics come out surprisingly good – none exceeds twice the average.

More statistics are available by going back to Model, selecting Options, and turning on (checkmarks) Matrix Measure and Correlation Matrices.


Turning on more options for report

Click OK and view the Results. On Bookmarks choose Matrix to see new statistics.

Matrix measures for design evaluation



Notice the condition number (12,220) far exceeds the level considered to represent severe multicollinearity for a design matrix (1000 or fewer). Viewing specific correlations reveals why.

Correlation matrices

Notice many values are highlighted in blue for being unacceptably high. No wonder Longley picked this data to test regression software!

Now, just for fun, press the Graphs button and select View, Perturbation (or press this option on the floating Graphs Tool).

Perturbation plot for standard error


Notice factors B and F exhibit the most dramatic tracks for standard error. On the floating Graphs Tool select 3D Surface. On the Factors Tool, right-click factor F:Time and change it to X1 axis.

3D view of standard error for factors B and F

There’s no sense doing anything more. By now it’s clear that this ‘design’ fails all the tests for a good experiment, but that’s generally the nature of the beast for happenstance data.




(Optimal Design) This tutorial details how Design-Expert® software crafts a response surface method (RSM) experiment within an irregular process space. If you are in a hurry to learn only the bare essentials of computer-based optimal design, then bypass the sidebars. However, they are well worth the time spent to explore things.

A food scientist wants to optimize a wheat product cooked at varying times versus temperatures. After a series of screening and in-depth factorial designs, the search for a process optimum has been narrowed to two factors, ranging as follows:


B. Time, 17 to 23 minutes.

However, it’s been discovered that to initiate desirable starch gelatinization, time must be at least 19 minutes when temperature is at 110 degrees C—the low end of its experimental range. On the other hand, when the temperature is increased to 180 degrees the starch will gel in only 17 minutes.

Constraint at lower levels of factors

To recap: At the lowest level of A, factor B must be at least 19, while at the lowest level of B, factor A must be at least 180. To complicate matters further, the experimenter suspects that the response surface may be wavy. That is, the standard quadratic model used for response surface methods (RSM) may fall short for providing accurate predictions. Therefore, a cubic model is recommended for the design.

A problem like this can be handled by Design-Expert via its constraint tools and optimal design capability.


Start Design-Expert and initiate the design process by clicking the blank-sheet icon on the left of the toolbar.

Click the Response Surface tab, select Optimal (custom) as the design, and enter the L[1] (lower) and L[2] (upper) limits as shown below.

110 180 A

23

17

B

19


Entering factor levels for optimal design

Create the multilinear constraint

Press the Edit Constraints button at the bottom of the screen.

Edit constraints fields for entering equations

At this stage, if you know the multilinear constraint(s), you can enter them directly by doing some math.

Explore the mathematics for multifactor constraints: Multifactor constraints must be entered as an equation taking

the form of: βL ≤ β1A+ β1B…≤ βU, where βL and βU are lower and upper limits, respectively. Anderson and Whitcomb

provide guidelines for developing constraint equations in Appendix 7A of their book RSM Simplified. If, as in this case

for both A and B, you want factors to exceed their constraint points (CP), this equation describes the boundary for the

experimental region:

A B

A A B B

A LL B LL1

CP LL CP LL

where LL is the lower level. In this case, lower levels are 110 for factor A and 17 for factor B. These factors’ CPs are

180 for factor A and 19 for factor B (the endpoints on the diagonal constraint line shown in the figure below).



Constraint points

Plugging in these values produces this equation:

A 110 B 17

1180 110 19 17

However, the software requires formatting as a linear equation. This is accomplished using simple arithmetic as shown

below.

A 110 B 171

70 2

70 2 1 2 A 110 70 B 17

140 2A 220 70B 1190

1550 2A 70B

Dividing both sides by 2 simplifies the equation to 775 A 35B . This last equation can be entered directly, or it can

be derived by Design-Expert from the constraint points while setting up a RSM design.

Using the Constraint Tool

Rather than entering this in directly, let’s make Design-Expert do the math. Click the Add Constraint Tool button. It’s already set up with the appropriate defaults based on what you’ve entered so far for the design constraints.

Constraints tool

110 180 A

23

17

B

19

CPA

CPB


As directed by the on-screen instructions, start by specifying the combination, or Vertex in geometric terminology, that must be excluded – shown in red below.

Vertex to be excluded

Click the downlist for vertex A and change it to 110.

Specifying the vertex coordinate for factor A

Then change the Vertex coordinate for B to 17.

Vertex coordinate B specified

The next step indicated by the instructions is to enter in the Constraint Point field the level for B (time) which becomes feasible when the temperature is at its low level of 110. Enter 19 – the time need at a minimum to initiate gelatinization of the starch.

Constraint point entered for factor B

Notice that Design-Expert correctly changed the inequality to greater than (>).

110 180 A

23

17

B

19



Now we turn our attention back to factor A (temperature) and its Constraint Point. It must be at least 180 when B (time) is at its vertex (low) level.

Constraint point set for A

Notice that Design-Expert previously changed the inequality to greater than (>), which is correct.

Press OK to see that Design-Expert calculated the same multilinear constraint equation that we derived mathematically.

Resulting multilinear constraint equation

Press OK to accept this inequality constraint. Then press Continue.

Building an optimal design

Leave the Search option default of “Best” and Optimality on the “I” criterion.

Explore details on optimal design options: Take a moment now to read the helpful information provided on screen,

which provide some pros and cons to these defaults. Note that we recommend the I optimal for response surface designs

like this. Press screen tips (light-bulb icon) for more details about the “Optimal Model Selection Screen.” Feel free to

click the link that explains how Design-Expert uses the “Best” search from coordinate and point exchange. Exit out (X)

of the tips for “Optimal Model Selection Screen.”

Press the Edit Model button to upgrade the default model of quadratic to Cubic. (Recall that the experimenter suspects the response might be ‘wavy’ — a surface that may not get fit adequately with only the default quadratic model.)


Cubic model specified

The defaults for the runs are a bit more than the experimenter would like to run for lack of fit and replicates. To limit the number of runs, change the To estimate lack of fit — from 5 (by default) to 4. Also decrease the Replicates to 4. The experimenter realizes this will decrease the robustness of the design, but decides that runs are too costly, so they need to be limited.

Lack of fit and replicates decreased

Press Continue to the response entry — leave this at the generic default of “R1” (we will not look at the experimental results). Then press Finish to build the optimal design for the constrained process space.

Explore how the program arrives at an optimal design: At this stage, Design-Expert randomly chooses a set of

design points called the “bootstrap.” It then repeatedly exchanges alternative points until achieving a set that meets the

specifications established in the model selection screen.

The optimal design (builds may vary due to random bootstraps)

Because of the random bootstraps, design builds vary, but all will be essentially (for all practical purposes) as optimal. Then to see the selection of points:

right-click the Select button and pick Design ID,

right-click the header for the Id column and Sort Ascending,

right-click the Select button and click on the Space Point Type.



You should now see a list of design points that are identified and sorted by their geometry. Due to the random bootstraps, your points may vary from those shown below.

Design layout sorted by ID with point type shown (your design may vary)

Explore the factor combinations selected by the program: Look over the design and, in particular, the point types

labeled as “vertex”: Is the combination of (110, 17) for (A,B) excluded? It should be – this is too low a temperature and

too short a time for the starch to gel.

P.S. To see how Design-Expert built your design, click the Summary node under the Design branch.

To accurately assess whether these combinations of factors provide sufficient information to fit a cubic model, click the design Evaluation node.

Design evaluation for cubic model

Based on the design specification, the program automatically chooses the cubic order for evaluation. Press Results to see how well it designed the experiment. (We went to View and turned off the Annotated Evaluation option before making the screen shot below, but you may want to keep this informative feature enabled.)


Results of design evaluation (yours may vary)

No aliases are found: That’s a good start! However, you may be surprised at the low power statistics and how they vary by term. This is fairly typical for designs with multilinear constraints. Therefore, we recommend you use the fraction of design space (FDS) graph as an alternative for assessing the sizing of such experiments. This will come up in just a moment.

Explore other aspects of design evaluation: Feel free to press forward to the leverage results and beyond by pressing

for this on the floating Bookmarks. Look over the annotations. Also press for Tips and look in Help for statistical

details.

Press the Graphs tab. Let’s assume that the experimenter hopes to see differences as small as two standard deviations in the response, in other words a signal-to-noise ratio of 2. In the floating tool for FDS Graph enter a d of 2. Press the Tab key to see the fraction of design space that achieves this level of precision for prediction (“d” signifying the response difference) given the standard deviation (“s”) of 1.



FDS graph

Ideally the FDS, reported in the legend at the left of the graph, will be 0.8 or more as it is in this case. This design is good to go!

Explore details on fraction of design space (FDS): A great deal of information on FDS is available at your fingertips

via the screen tips. Take a look at this and click the link to the “FDS Graph Tool.” Exit out of the screen-tips help

screen.

On the floating Graphs Tool press Contour to see where your points are located (expect these to differ somewhat from those shown below).

Standard error plot

Our points appear to be well-spaced and the choices for replication (points labeled “2”) seem reasonable. Select View, 3D Surface (or choose this off the Graphs Tool) to see the surface for standard error.


Response surface for standard error

This design produces a relatively flat surface on the standard error plot, that is, it provides very similar precision for predictions throughout the experimental region.

This concludes the tutorial on optimal design for response surface methods.

Explore what happens if you modify the design: You are now on your own. Depending on which points Design-

Expert picked for your design, you may want to add a few more runs* at the extreme vertices, along an edge, or in other

locations to decrease relatively high prediction variances. If you modify the design, go back and evaluate it. Check the

3D view of standard error and see how much lower it becomes at the point(s) you add. In any case, you can rest assured

that, via its optimal design capability, Design-Expert will provide a good spread of processing conditions to fit the model

that you specify, and do it within the feasible region dictated by you.

*We detailed how to modify the design layout in Part 2 of General One-Factor Tutorial (Advanced Features).

DX9-05-5-MultifactorRSM-Cat-V8.docxx Rev. 5/2/14

Design-Expert 8 User’s Guide Categorical RSM Tutorial 1


(Adding Categorical Factors)

Combining categorical with numeric factors

Procter and Gamble (P&G) engineers working on a new sealing process became concerned about how maximum peel strength would be affected by changing suppliers of packaging material (Brenneman, William A., and William R. Myers, “Robust Parameter Design with Categorical Noise Variables,” Journal of Quality Technology 35, no. 4 (October 2003): 335-341). They set up an RSM design to vary several key factors on the sealing machine, including the supplier:


B. Pressure, 2.2 to 3.2 Bar.

C. Speed, 32 to 50 cpm (cycles per minute).

D. Supplier: S1, S2, S3.

Due to limitations of time and other resources, a maximum of 37 runs can be performed. Therefore, simply conducting a standard central composite design (CCD) or Box-Behnken design (BBD) for each of the three suppliers will not do – these design choices produce far too many runs (60 and 51, respectively). Instead, P&G engineers use a D-optimal design.

The data below come from a simulation loosely based on the predictive model reported in the cited article. (For tutorial purposes, some liberties are taken to make the procedural outcome more challenging.) Assume the maximum peel strength ideally hits a target of 4.5 pound-force (lbf). However, it must exceed 3 lbf to prevent leaking and not exceed 6 lbf because the package becomes too difficult to open.

This tutorial explains how a design is constructed via D-optimal criteria with added points for lack of fit testing and pure-error estimation. Notice we use nearly all budgeted runs. Also, aided by a feature in the software that forces balance, we make sure each supplier receives an equal number of design runs.

With guidance from the tutorial you analyze responses and then find desirable solutions meeting the following objectives – listed according to relative acceptability for all concerned:

1. Can the process be adjusted to attain a maximum peel strength target of 4.5 lbf for any one supplier?

2. The purchasing agent intends to break down suppliers as follows: 50% to S1, 25% to S2, and 25% to S3. Can you find a process setup that works for all suppliers? If not, a two-supplier option might be satisfactory, provided S1 is one of them. In other words, assuming the process is not robust for all three suppliers, can the process be set up in a manner that either S1-S2 or S1-S3 meets specifications?

2 Categorical RSM Tutorial Design-Expert 8 User’s Guide

3. If it is not possible to achieve a common process for even two of the three suppliers, perhaps the upper limit of 6 can be raised on the maximum peel strength specification. (Assume customers are willing to use scissors or, better yet, package designers notch the plastic for easier opening.) Does this open a window of operability for multiple suppliers at one set of process conditions?

Here is the simulation data loosely based on the above cited article:

Std A:

Temp

(deg C)

B:

Pressure

(Bar)

C:

Speed

(cpm)

D:

Supplier

Peel

Strength

(lbf)

1 193 2.2 32 S1 4.6 2 230 2.2 32 S1 10.0 3 193 3.2 32 S1 7.2 4 193 3.2 32 S1 6.6 5 230 3.2 32 S1 5.5 6 202.25 2.7 41 S1 8.2 7 230 2.7 41 S1 8.5 8 193 2.2 50 S1 6.7 9 230 2.2 50 S1 11.0

10 230 2.2 50 S1 11.0 11 230 3.2 50 S1 7.8 12 230 3.2 50 S1 7.1 13 230 2.2 32 S2 5.1 14 230 2.2 32 S2 6.7 15 211.5 2.7 32 S2 2.9 16 230 3.2 32 S2 1.7 17 211.5 2.2 41 S2 6.3 18 193 2.7 41 S2 2.0 19 220.75 2.95 41 S2 3.7 20 193 2.2 50 S2 2.4 21 230 2.7 50 S2 4.0 22 230 2.7 50 S2 4.1 23 193 3.2 50 S2 4.8 24 193 3.2 50 S2 5.1 25 193 2.2 32 S3 6.7 26 193 2.2 32 S3 6.7 27 230 3.2 32 S3 8.0 28 230 3.2 32 S3 7.1 29 220.75 2.45 36.5 S3 9.2 30 202.25 2.7 36.5 S3 8.1 31 211.5 3.2 41 S3 7.2 32 230 2.2 50 S3 9.5 33 230 2.2 50 S3 9.8 34 211.5 2.7 50 S3 6.2 35 193 3.2 50 S3 5.9 36 230 3.2 50 S3 4.8

Data based on package sealing experiment




Start Design-Expert® software and initiate your design process by clicking the blank-sheet icon on the left of the toolbar, or select File, New Design, or simply click New Design on our opening screen.

New easy-start design options

Click the yellow Response Surface folder tab, select Optimal as your design. For Numeric Factors choose 3 from the droplist. For Categoric Factors choose 1. Key in (and tab over) Name, Units, L[1] (lower limit) and L[2] (upper limit) for numeric factors A, B, and C as shown below. For categoric factor D, key in (and tab over) Name, Units, Levels (3), and L[1], L[2], L[3].

Entering numeric and categoric factor levels for optimal design

In the above screen shot, the Type column automatically selects via default either “Nominal” or “Continuous.” “Nominal” applies to categoric factor D because its treatments are discrete names (S1, S2, S3). When entering ordered treatments (1, 2, 3, for example), it is best to set these up with “Ordinal” contrasts – available as a droplist option – because subsequent model construction and ANOVA layout are then properly applied.

Press Continue. Notice the “Search” droplist defaults to “Best” and “Optimality” defaults to “IV.”


Default selections for optimal design

To create more convenient levels in the numerical factors, change the Search to Point exchange.

Changing search to point exchange

Change the Optimality to D, which may be the better choice for designs with categorical factors as noted in the comments you see on screen. For more statistical details on Design-Expert’s optimal model selection screen press the screen tips icon (the light bulb) or select Tips, Screen Tips off the main menu.

Changing to D optimality

At this stage the runs total to 28, which is not divisible by 3 – desirable for splitting evenly among your three suppliers. To compensate for this, bulk up your design to 36 runs (12 to each supplier) by increasing To estimate lack of fit to 9. Tab to Replicates and again enter 9. Set a check-mark at Force categoric balance. Not doing this could result in a design with more of one supplier than another, which would doubtless generate questions from the purchasing department because it would seem unfair. A warning alerts you that forcing categoric balance might require adjustments.



Forcing categoric balance might require adjustments

Press OK to continue. Now press Edit candidate points so you can see which combinations Design-Expert starts with, from which only an optimal subset will be chosen. A pop-up screen appears. Press Calculate points to see if the point selection produces more than the minimum needed to fit the selected model.

Optimal specifications modified

The program now identifies 53 candidate points for each of the 3 suppliers for a total of 159, from which it will select optimally 18 points needed at a minimum for the quadratic model. Design-Expert will add 9 more runs with unique factor combinations to test lack of fit. Finally, 9 of the 27 points already identified will be replicated.

Press OK, then press Continue. Enter response Name and Units as shown below.

Response names and units – name field expanded horizontally


Press Continue again. A warning may alert you Design-Expert is preserving a balanced design for you, which causes a slight loss in optimality.

Forced-balancing warning about a slight loss in optimality

Your loss percentage may differ from that shown below, but in any case it will be negiligible. If the warning appears, choose Yes to bring up the design template.

Analyze the Response

Your layout may match what’s shown in the original data table, but there are actually many possible and equally good point selections. To avoid the possibility of a mismatch, but more importantly, to save you time in entering responses, click the open design icon (the one that looks like a folder that’s shown below) or select File, Open Design.

Opening a design

Find the file named Peel.dxp and open it. If you are asked to save changes to “MyDesign.dxp”, choose no.

Opening the file previously created for this case



Notice that the header for the Response 1 column is truncated (Max peel str…). To expand the width of the column automatically, just mouse over the right border until you get the double sided arrow (shown below) and double-click. Now, the whole column header (Max peel strength) can be seen.

Automatically adjusting column width using the double arrow

Under the Analysis branch, click Max peel strength. Then press Fit Summary atop the screen.

Fit summary report

Notice that design-for model, quadratic, is suggested. On the Bookmarks palette, click Lack of Fit. This looks good (insignificant at p>0.1), so press ahead to the Model button, then ANOVA with this chosen model.


ANOVA for quadratic model

The statistics are excellent. Some terms come out insignificant, but carry them along by pressing Diagnostics (look these over – they shouldn’t alarm you). Now view Model Graphs. This is where it gets intriguing because the suppliers and purchasing agent are anxious to see the findings! Select View, Perturbation (or click this option on the floating Graphs Tool) to see an overview plot before generating contours and 3D surfaces.



Perturbation plot

Remember that 4.5 lbf is the target, so supplier S1 – chosen by default – is coming out high. Lowering factor C will help bring the response down towards the target value. Return to View, Contour (or click this plot option on the Graphs Tool). On the Factors Tool, click C:Speed and slide it left. At the lowest response levels – the defaulting blue-green graduated color shading – see what’s predicted via a right-click and Add flag.

Supplier 1 is not able to produce the target response, even at lowest speed (factor C)


On the Factors Tool at D:Supplier, click the middle dot to see S2 results. Then right-click the graph, Add Contour, and drag it near the required 4.5.

Supplier 2 contour plot

On the Factors Tool, click C:Speed and slide it right. Then right-click and Add

flag near the middle that achieves the response specification. This second supplier (S2) looks much better. With their material, the process need not be pushed too hard in one factor direction or another to meet the critical specification of maximum peel strength on their packaging material.

Supplier 2 is looking good!

On the Factors Tool at D:Supplier, click the right dot to see S3 results – it doesn’t look good – the peel strength goes too high again. Select from the Graphs Tool the Perturbation plot to get an overview of what’s occurring with this supplier. Click Default on the Factors Tool to center C:Speed. Then click the right dot at D:Supplier to reset it for S3.



Perturbation plot for supplier S3

Notice the high factor C level might allow the 4.5 target to be achieved at certain A and B settings. Go back to View, Contour, and on the Factors Tool click C:Speed and slide it right. Then right-click the graph and select Graph preferences.

Third supplier chosen for contour plot with Graph preferences selected

On the Contours tab for graph preferences, select Incremental option. Enter a Start of 0.5, Step of 0.5, and set Levels at 20 (the maximum). Then change Format to 0.0.


Changing contour levels

On the Surface Graphs tab, select for Contour graph shading the Std Error shading option (better for black and white printing).

Standard error shading selected for contour graph

Press OK. Click the 4.5 contour line and note where the specification is met by this supplier with the proper process setup. It becomes a red line for easy viewing and tracking.

The contrast on the graph is not sufficient to see what’s going on, so right-click on the shading bar in the legend. In the Edit Legend dialogue box (see picture below) change the high value to 0.9. This will cause the grey shading to get darker at a lower level, creating more contrast. This trick can be used with a color contour



plot, too. Adjusting the high and low levels on the shading bar can create better looking plots with more colors.

The target of 4.5 highlighted as a contour for supplier 3. Right-click the shading bar to edit the legend and change the high to 0.9

The red dots represent conditions where actual experimental runs were performed and, in one case, replicated (as indicated by the “2”). Unfortunately, no points are seen near the desired contour. Thus this region is shaded darker by Design-Expert to indicate a higher standard error for predictions.


Recall the questions asked at the outset of this tutorial about the new packaging machine developed by the engineers at P&G. We now know the answer to the first one – yes, the process can be adjusted to hit the maximum peel strength target of 4.5 lbf for one or more suppliers.

The purchasing agent intended to break down the supply in this manner: 50% to S1, 25% to S2, and 25% to S3. This led to the second question: Can you find a process setup that works for all suppliers? The contour plots for S1 make it clear that this supplier, despite being favored by the purchasing agent, will not achieve targeted peel strength within current factor levels. That foils the next most acceptable alternative of a two-supplier option with S1 being one of them. (For some reason, hopefully based on cost, the purchasing agent really likes the first supplier!)

All that’s left for the engineers is the following fallback position: If it is not possible to achieve a common setup for even two of the three suppliers, perhaps the upper limit of 6 could be raised on the maximum peel strength specification. (Assume your customers are willing to use scissors or, better yet, package designers can notch the plastic for easier opening.) Will this open a window of operability for multiple suppliers using one set of process conditions?


It’s time to explore the operating windows of each of the three suppliers for making packaging that exceeds 3 lbf (to prevent leaking) and not exceed 6 lbf (making the package too difficult to open). Under the Optimization branch, click the Graphical node. Enter the Limits at Lower 3 and Upper 6.

Criteria for graphical optimization

Click the Graphs button. You’ll see there’s no operating window at the default settings. On Factors Tool you must therefore click the bar for C:Speed and slide it left, thus opening a sweet spot for favored supplier (S1).

Sweet spot for supplier 1 at low speed (C)

Next click D:Supplier dots for alternatives S2 (middle) and S3 (right). Notice only S2 works at this level of C, but it offers no window of opportunity co-existing with S1.



Window for supplier 2 at same level of factor C

Now the only remaining alternative is to click the D:Supplier dot for S1 (left dot), then click and drag the border for Max peel strength to the right, causing it to increase above 6. Go ahead and push it all the way up to near 7. (Better have your scissors handy for opening the package!)

Dragging the border up to 7 for max peel strength

That’s far enough into this case study for tutorial purposes. If you like, click through the other suppliers to see if there’s a common condition where even the newly raised packaging specification can be achieved. The answer may surprise you.

Z:\Manual\DX9\DX9-05-6-VerRuns-FT.docx 5/2/2014 9:41:00 AM

Design-Expert® software version 9 (DX9): How to add verification runs

A new feature added to Design-Expert 9 is the ability set the status of a run to “Verification”. A verification run is part of the original experiment but the information the run generates is not used to fit a model for the response. The run is just used to check the model predictions, either by visual inspection of the fit on the graph or by some other quantitative metric.

Method A: Manual Setup

The simplest way to add verification runs is to type them in and change the row status to "Verification". This might be done to verify customer expectations or standard operating procedures. To create a blank line in the design layout, right-click the button to the left of any row and choose Insert Row as shown below. (To try this for yourself, open the “RSM-a.dxpx” tutorial file.)

This adds a blank row where you can enter the desired verification settings. After entering the factor levels, right-click on the row header again and Set Row StatusVerification as shown below.

Once converted to verification status, the row font changes to bold and the run number goes into brackets []. Design-Expert guards against mistaken entry of extreme factors settings. In any case, factor settings within the original experimental limits are advised (be careful of extrapolation!). Responses from verification runs appear as check-marked () squares on the graphical output as seen to the right.

Method B: Computer-Generated to Fill Gaps

Augmenting a design with lack-of-fit points provides a more sophisticated way to add verification runs. Do so by choosing Design Tools Augment Design Augment. This will bring up the Optimal Augment window. Near the middle of this window, change the Block to Day 2. Change the number of Lack-of-fit points to the desired number of verification runs—in this case 1, which when added in this way are

Z:\Manual\DX9\DX9-05-6-VerRuns-FT.docx 5/2/2014 9:41:00 AM

selected to be space filling, that is, they will fill the gaps in the design space. Hit the tab key to update your screen to match the one shown below. Then click OK to add the runs.

Runs added through the augment procedure will always be added to the bottom of the block. Once you have added the new runs, just right-click on the row headers for those runs and choose Set row statusVerification to turn these into verification runs.

Rather than putting verification runs at the end blocks, considers randomizing them into the design. Do so via a right-click on the Run column and selecting “Randomize”. Then you can re-randomize any or all blocks in your design.

Summary

When modeling the responses, DX9 ignores verification runs. However, so experimenters can see how well their model fits, verification runs show up on Graph Columns, the Diagnostic Influence report, and the Model Graphs.

Z:\Manual\DX9\DX9-05-7-ConfRuns-FT.docx 5/2/2014 9:41:00 AM

Design-Expert® software version 9 (DX9) Confirmation Runs Feature DX9 allows direct entry of confirmation runs for model validation. To see this demonstrated, open

“RSM-a.dxpx”. Then, as shown below, under the

Optimization branch, do a Numerical optimization

to target the Activity at 63 within specification

Limits of 60 and 66. Click the Solutions tab to see

the result of Design-Expert’s search for the most

desirable combination of factors.

Now, click the Confirmation node and for n = 5

check () the Enter Data box and, in the

spreadsheet that pops open, type the Activity

results shown below:

The mean then appears in the middle of the prediction interval—black if OK, but red when out of

bounds. How did you do?

DX9-06-1-Mix.docx Rev. 5/2/14

Design-Expert 9 User’s Guide Mixture Tutorial 1

Mixture Design Tutorial

(Part 1/2 – The Basics)

Introduction

In this tutorial you are introduced to mixture design. If you are in a hurry to learn about mixture design and analysis, bypass the sidebars. However, if/when you can circle back, takes advantage of these educational sidetracks.

Explore more fundamental features of the software: Mixture design is really a specialized form of response surface

methods (RSM). To keep this tutorial to the point, we will not go back over features detailed in the RSM tutorials so

you’d best either work through these first, or do so after completing this one on mixtures. Otherwise you will remain

ignorant of many useful features and miss out on important nuances in interpreting outputs.

To gain a full working knowledge of this powerful tool, we recommend you attend our workshop on Mixture Design for Optimal Formulations. Call Stat-Ease or visit our website at www.statease.com for a schedule. For a free primer on mixture design, go to the Stat-Ease home page and follow the link that says “I’m a formulator.” If you seek statistical details on this topic, see John Cornell’s Experiments with Mixtures, 3rd edition, published by John Wiley and Sons, New York, in 2002.

This tutorial demonstrates only essential program functions. For more details, check our extensive Help system, accessible at any time by pressing F1. Its hypertext search capability makes it easy for you to find the information you need.

The Case Study – Formulating a Detergent

A detergent must be re-formulated to fine-tune two product attributes, which are measured as responses from a designed experiment:

Y1 - viscosity

Y2 - turbidity.

Three primary components vary as shown:

3% ≤ A (water) ≤ 8%

2% ≤ B (alcohol) ≤ 4%

2% ≤ C (urea) ≤ 4%

These components represent nine weight-percent of the total formulation, that is:

A + B + C = 9%

Other materials (held constant) make up the difference: 91 weight-percent of the detergent. For purposes of this experiment they are ignored.

Experimenters chose a standard mixture design called a simplex lattice. They augmented this design with axial check blends and the overall centroid. Vertices and overall centroid were replicated, increasing the experiment size to 14 blends total.


2 Mixture Tutorial Design-Expert 9 User’s Guide

Augmented simplex lattice (second degree)

This case study leads you through all the steps of design and analysis for mixtures. The next tutorial, Part 2, instructs how to simultaneously optimize the two responses.


Start the program by finding and double clicking the Design-Expert software icon. Take the quickest route to initiating a new design by clicking New Design on the opening screen as shown below.

Opening screen – New Design highlighted

Click the Mixture folder tab. The design you want, a simplex lattice, comes up by default.

Choosing a mixture design

90

50

70

30

10

90

50

70

30

10

90

50

70

30

10

X1

X2 X3

2

2

2

2



Explore Screen Tips and Help on mixture design options: Some on-screen details appear, but more are available by

pressing the screen tips button (or via Tips, Screen Tips). Check this out! Close Tips by pressing X at its upper-right

corner. Next explore Help, Contents. Double-click Mixture Designs, then Mixture Design Choices. After looking

over all this helpful information, close Help by pressing X.

Now change the number of Mixture Components to 3. Enter components and their limits as shown below in all Name, Low, and High fields, pressing the Tab key after each entry. Enter 9 in the Total field and % as your Units.

Entering components, limits, and total

Press Continue. Immediately a warning appears.

Adjustment made to constraints

Press OK. Notice that although you entered the high limit for water as 8%, Design-Expert adjusts it to 5% — leaving room for 2% each of the other two ingredients within the 9% total. Otherwise, at 8%, water and the low levels of alcohol and urea would total 12%. Design-Expert recognizes that this does not compute. Very helpful!

Explore optimal design: For many mixture designs you may be warned at this point that it cannot fit a simplex.

Warning that comes up if mixture region is not a simplex

You should then shift to the Optimal design choice. See this option detailed in the Multifactor RSM Optimal tutorial.

Press Continue and the software’s adjustment lets you move on. Now you must choose the order of the model you expect is appropriate for the system being studied. In this case, assume that a quadratic polynomial, which includes second-order terms for curvature, will adequately model the responses. Therefore, leave the order at Quadratic. Keep the default check-mark at “Augment design” but change Number of runs to replicate to 3. Press your Tab key to display the correct number of total runs.


Simplex-lattice design form (after pressing Tab key)

By keeping (accepting) the “Augment design” check-mark, you allow Design-Expert to add the overall centroid and axial check blends to the design points.

Explore why the experiment reduced the replicates: The “Number of runs to replicate” field, which had defaulted to

4, causes the specified number of highest leverage experiments to be duplicated. In this case, there are three points with

highest leverage – the vertices of the triangular simplex. This makes the fourth replicate a bit awkward because it creates

an imbalance in the design. Feel free to try this and see for yourself. Then rebuild the design saying “Yes” to Use

previous design info” — thus preserving your typing of component names, etc.

Press Continue to proceed to the next step in the design process. In the Responses droplist, choose 2. Then enter all response Names and Units as shown below.

Response entries

Up to now you’ve been able to click the Back button at the lower right of your screen and move through the design forms to change requirements. When you press Finish on this page, Design-Expert completes the design setup for you.

Modify the Design

To top off this experiment design let’s replicate the centroid. In the Design layout right click the Select column header at the upper-left corner and pick Design ID. Go back and alsso Select (display) the Space Point Type column. This is very helpful for insights about design geometry.

Adding columns to design layout via the Select option (right-click menu)

Next, right click the column header labeled Id, and select Sort Ascending. Now your screen should match that below except for the randomized run numbers.



Initial design sorted by ID with point type shown (run order randomized)

The experimenters ran an additional centroid point, so in the box to the left of Id 0 (point type = “Center”) right-click and select Duplicate.

Duplicating the centroid

Whenever you insert, delete, or duplicate rows, always right-click the Run column-header and chose Randomize.

Re-randomizing the run order

A box pops up asking if you’d like all blocks to be randomized. In this case there is only one block, so simply press OK. Notice that run numbers now change.

Again, right-click the Run column-header, but this time choose Sort Ascending.

Explore what you would do next if this were your experiment: Normally you’d now do a File, Print to produce a

‘recipe’ sheet to run your experiment. Go ahead if you wish or simply do a File, Print Preview.


Save Your Work

Because you’ve invested time into your design, it is prudent to save your work. Click File then Save As. The program displays a standard file dialog box. Use it to specify the name and destination of your data file. Enter a file name in the field with default extension dxpx. (We suggest tut-mix.) Click Save.

Analyze the Results

Assume your experiments are completed. You now need to enter responses into the Design-Expert software. For tutorial purposes, we see no benefit to making you type all the numbers. So to save time, read the response data in from a file that we’ve placed in your program’s Data directory. Select File, Open Design. Click the file named Mix.dxpx. Press OK. You now should be displaying the response data shown below. (Note the design layout returns to the default selection, which we have not changed.)

There’s no need for typing in this case, but normally you’d have invested much more work by this stage, so click File then Save to preserve all response data.

Ready-made response data

Go to the Analysis branch of Design-Expert and click the Viscosity node. Let’s progress through the tabs atop the window.



First step in analysis: Transformation options

First, consider doing a transformation on the response. In some cases this improves the statistical properties of the analysis. For example, when responses vary over several orders of magnitude, the log scale usually works best. In this case the ratio of maximum to minimum response is only a bit over 4, which isn’t excessive (see detail at bottom of the screen), so leave the selection at its default, None, because no transformation is needed. Also, leave the coding for analysis as pseudo because this re-scales the actual component levels to 0 – 1.

Explore details on coding for mixture models: For complete details on pseudo and other coding for mixture models,

see the textbook by Cornell mentioned at the outset of this tutorial. In the meantime, bring up Help, Contents. Then on

the Contents tab double-click Mixture Designs and Mixture Statistical Information. Select Component Scaling in

Mixture Designs. After studying all the information you find here, close Help by pressing X.

Next click the Fit Summary tab. Here Design-Expert fits linear, quadratic, special cubic, and full cubic polynomials to the response.


Fit summary reports

Explore re-sizing column widths: You may need to enlarge some columns horizontally so all outputs appear in full.

If so, simply place the cursor at any column-header’s left edge until the cursor changes to a double-ended arrow. Drag it

to the right.

To begin your analysis, look for any warnings about aliasing. In this case, the full cubic model and beyond could not be estimated by the chosen design — an augmented simplex design. Remember, you chose only to fit a quadratic model, so this should be no surprise.

Next, pay heed to the model suggested by Design-Expert in the first table at the top, which re-caps what’s detailed below.

Now on the floating Bookmarks press forward to the Sum of Squares breakdown.

Sequential models sum of squares table



Analysis proceeds from a basis of the mean response. This is the default model if none of the factors causes a significant effect on the response. The output then shows the significance of each set of additional terms. Notice that below the level where the program says “Suggested” the p-value become insignificant (>0.05), thus there is no advantage to adding further terms.

Explore more details on the breakdown of model sums of square: Here’s a line-by-line detailing:

- “Linear vs Mean”: the significance of adding the linear terms after accounting for the mean. (Due to the constraint

that the three components must sum to a fixed total, you will see only two degrees of freedom associated with the

linear mixture model.)

- “Quadratic vs Linear”: the significance of adding the quadratic terms to the linear terms already in the model.

- “Sp Cubic vs Quadratic”: the contribution of the special cubic terms beyond the quadratic and linear terms.

- “Cubic vs Sp Cubic”: the contribution of the full cubic terms beyond the special cubic, quadratic, and linear terms.

(In this case, these terms are aliased.)

- And so on….

For each set of terms, probability (“Prob > F”) should be examined to see if it falls below 0.05 (or whatever statistical

significance level you choose). Adding terms up to quadratic significantly improves this particular model, but when you

get to the special cubic level, there’s no further improvement. The program automatically underlines at least one

“Suggested” model. Always confirm this suggestion by reviewing all tables under Fit Summary.

On the floating Bookmarks tool click Lack of Fit to move on to the next table. This table compares residual error with pure error from replication. If residual error significantly exceeds pure error, then deviations remain in the residuals that can be removed using a more appropriate model. Residual error from the linear model shows significant lack of fit (this is bad), while quadratic, special cubic, and full cubic do not show significant lack of fit (this is good).

Lack of fit table

At this point the quadratic model statistically looks very good indeed.

Now on the floating Bookmarks tool click R-Squared to view the bottom table: “Model Summary Statistics.” Here you see several comparative measures for model selection.


Summary statistics

Ignoring the aliased cubic model, the quadratic model comes out best: low standard deviation (“Std Dev”), high “R-Squared” statistics, and low “PRESS.”

Before moving on, you may want to print the Fit Summary tables via File, Print. These tables, or any selected subset, can be cut and pasted into a word processor, spreadsheet, or any other Window application. You’re now ready to take an in-depth look at the quadratic model.

Model Selection and Analysis of Variance (ANOVA)

Click the Model tab atop the screen to see the model suggested by Design-Expert software.

Choosing the model

Explore how you can over-ride the model suggested by the program: Press the screen tips button to see very

helpful information about what you can do at this stage. You may select models other than this defaulting quadratic

model from the pull down list. (Be sure to do this in the rare cases when Design-Expert suggests more than one model.)

On the current screen you are allowed to manually reduce the model by clicking off terms that are not statistically

significant. For example, in this case, you will see in a moment that the AB term is not statistically significant.



Also, as noted under the “Model Reduction” subheading appearing here in the Tips screen, Design-Expert provides

several automatic reduction algorithms as alternatives to “Manual” displaying in the Selection droplist: Backward,

Forward, and Stepwise. Click the droplist’s down arrow if you’d like to try one. You will see a recommendation pop up

on what works best as a general rule. However, we recommend you not reduce mixture models unless you’re sure, based

on statistical and subject-matter knowledge, that this makes sense. If you really want to be competent on this, attend our

Mixture Design for Optimal Formulations workshop.

Close Tips by clicking X.

Press the ANOVA button for details about the quadratic model.

ANOVA report with annotations on

The statistics look very good.

Explore details on the ANOVA results: The model has a high F value and low probability values (Prob > F). That is

good as you will infer from the annotation provided by Design-Expert. The probability values show the significance of

each term.

P.S. Because the mixture model does not contain an intercept term, the main effect coefficients (linear terms) incorporate

the overall average response and are tested together.

Use Bookmarks to jump to the next report – R-Squared statistics.

R-squared and other statistics after the ANOVA


These statistics, many of which you’ve already seen in the “Model Summary Statistics” table, all look good. Note the more than adequate precision (Adeq Precision) value of 27.943.

Next, view the coefficients and associated confidence intervals for the quadratic model.

Coefficients for the quadratic model

Explore alternative equations for response prediction: Continue further to see several alternative models that vary

by how components are coded. The annotations provide ideas on how they differ.

Continue on to the next tab from ANOVA — Diagnostics.

Diagnose the Statistical Properties of the Model

The most important diagnostic, the normal probability plot of the residuals, comes up by default.

Normal Probability Plot of Residuals

The data points should be approximately linear. A non-linear pattern (such as an S-shaped curve) indicates non-normality in the error term, which may be corrected by a transformation. There are no signs of any problems in our data.

Explore the form of residuals: At the left of the screen you see the floating Diagnostics Tool palette. Be aware

that residuals are externally studentized unless you elect otherwise (not advised). Studentization counteracts varying

leverages due to design point locations. For example, center points carry little weight in the fit and thus exhibit low

leverage. Externalizing the residuals isolates each one in comparison to the others so discrepant results stand out more.

Now click the Influence option. To bring up bring up case-by-case details on many of the statistics shown graphically for diagnostic purposes: Press Report.



Diagnostics tool and report

Notice that one value is flagged in red (and with an asterisk) for exceeding suggested limits: DFFITs for standard order 11. As we discussed in the General One-Factor Tutorial (Part 2 – Advanced Features), this statistic stands for difference in fits. It measures change in each predicted value that occurs when that response is deleted. Given that only this one diagnostic is flagged, it probably is not a cause for alarm. However, observe that it’s one of the highest viscosity responses (Actual Value = 130.00), so the experimenter might want to double-check the accuracy of this response.

Explore all the diagnostic graphs: Check out all the graphs on either side of the tool if you like. Explanations for

most of these graphs are addressed in earlier tutorials. Get more details via Screen Tips and Help. In this case, none of

the graphs, excepting DFFITS, indicates any cause for alarm.

Examine Model Graphs

The residuals diagnosis reveals no statistical problems, so now let’s generate response surface plots. Click the Model Graphs tab. The 2D contour plot comes up by default in graduated color shading.

Response surface contour plot


Note that Design-Expert displays any actual point included in the design space. In this case you see a plot of viscosity as a function of the three mixture components. This slice includes two centroids as indicated by the red dot and the number “2” at the middle of the contour plot.

Explore all the Factors Tool: The Factors Tool displays along with the default plot. Move this floating tool as

needed by clicking on the top blue border and dragging it. The tool controls which factor(s) are plotted on the graph.

The Gauges view is the default. Each component listed has either an axis label, indicating that it is currently appearing

on the graph, or a red slider bar, which allows you to choose specific settings for those not currently plotted. This case

study involves only three components, all of which fit on one mixture plot – a ternary diagram. Therefore, you do not

see any red slider bars. If you did, they would default to the midpoint levels of the components not currently assigned to

axes. You could then change a level by dragging the red slider bars left or right. If you’d like to see a demonstration of

this feature, work through the Multifactor RSM Tutorial (Part 1 – The Basics).

Place your mouse cursor over the contour graph. Observe how it turns into a cross (+).

Then notice in the lower-left corner of the screen that Design-Expert displays the predicted response and coordinates.

Coordinates display at lower-left corner of screen

To enable a handier tool for reading coordinates off contour plots, go to View, Show

Crosshairs Window.

Showing crosshairs window

Now move your mouse over the contour plot and notice that Design-Expert generates the predicted response for specific values of the factors that correspond to that point. If you place the crosshair over an actual point, for example, the check-blend midway between the centroid and the upper vertex (corner labeled “A”), you also see the observed value (in this case: 35.100) as shown below.



Prediction where an actual run was performed

Explore all the Crosshairs capability: Press the Full button to see confidence and prediction intervals in addition to

the coordinates and predicted response, as shown above right.

Full Crosshairs display

Close the crosshairs window by clicking X.

Zooming In and Out

Let’s say you’re interested in highest values for viscosity. With your left mouse button held down, drag over the lower right corner of the contour graph.

Corner identified for zoom

Now the area you chose is magnified.


Zoomed-in area on contour plot

To revert to the full triangle plot, right-click anywhere over the plot and select Default View Window.

Default view window

Explore how to make the contour plot more presentable: Many more features are available to modify the look of

contour plots. These are detailed in the Multifactor RSM Tutorial (Part 3 – Advanced Topics) Tips and Tricks for

Making Response Graphs Most Presentable.

Trace Plot

Wouldn’t it be handy to see all your factors on one response plot? You can do this with the trace plot, which provides silhouette views of the response surface. The real benefit from this plot is for selecting axes and constants in contour and 3D plots. From the floating Graphs Tool select Trace. Trace plots show the effects of changing each component along an imaginary line from the reference blend (defaulted to the overall centroid) to the vertex. For example, click on the curve for A and it changes color.



Trace plot – component A highlighted

Notice that viscosity (the response) is not very sensitive to this component.

Explore options for trace directions: In this case, where the experimental region forms a simplex, it matters little

which direction you take. Check this out by going to the Trace Graph tool and pressing Cox. In the Cox direction, as

the amount of any component increases, the amounts of all other components decrease, but their ratio to one another

remains constant. Chemists may like this because it preserves the reaction stoichiometry. However, when plotted in this

direction, traces for highly constrained mixture components (such as a catalyst for a chemical reaction) become

truncated. Thus, mixture-design experts argue that although it no longer holds actual ratios constant, Piepel’s direction

provides a more helpful plot by providing the broadest coverage of the experimental space. For this reason Piepel is the

preferred plot in Design-Expert. For more detail, search in Help for “trace plot”.

P.S. Trace plots depend greatly on where you place the starting point (by default the centroid). See for yourself by

moving slide bars on the Factors Tool. When you are done, press the Default. Consider that the traces are one-

dimensional only, and thus cannot provide a very useful view of a response surface. A 3D response plot provides a

better picture of the surface, and ultimately provides the basis for numerical optimization. It’s the ultimate tool for

determining the most desirable mixture composition.

If you experiment on more than three mixture components, use the trace plot to find those components that most affect the response. Choose these influential components for the axes on the contour plots. Set as constants those components that create relatively small effects. Your 2D contour and 3D plots will then be sliced in ways that are most visually interesting.

Explore this heads-up on how to deal with more than three components: When you have more than three

components to plot, Design-Expert software uses the composition at the optimum as the default for the remaining

constant axes. For example, if you design for four components, the experimental space is a tetrahedron. Within this

three-dimensional space you may find several optimums, which require multiple triangular “slices,” one for each

optimum.

Generating a 3D View of the Response Surface

Now to really get a feel for how response varies as a function of the two factors chosen for display, select View, 3D Surface. A three-dimensional display of the response surface appears. If coordinates encompass actual design points, these emerge.


3D response surface plot

You can rotate the 3D plot directly by grabbing it with your mouse. It turns into a hand when placed over the graph. Then click and hold the left mouse-button and drag. Try it! What’s really neat is how it becomes transparent so you can see hidden points falling below the surface.

Transparent view as surface is rotated, which allows points to be seen better

When you’re done spinning the graph around, right click over it and select preferences Default rotation.



Right-click menu for 3D graph

The graph then re-sets to its original settings.

Explore another way to rotate graphs: Select View, Show Rotation for a tool that makes it easy to view 3D surface

plots from any angle.

Control for rotating 3D plot

Move your cursor over the tool. The pointer changes to a hand. Now use the hand to rotate the vertical or horizontal

wheel. Watch the 3D surface change. It’s fun! Notice that you can specify precise horizontal (“h”) and vertical (“v”)

coordinates. Give this a try, too. Then press Default and X off the view of Rotation tool.

Design-Expert offers many options for 3D graphs via its Graph Preferences, which come up with a right-click over the plot. For example, if you don’t like graduated colors, go to the Surface Graphs tab and change 3D graph shading to wire frame view (a transparent look).

Response Prediction

Response prediction in Design-Expert software falls under the Post Analysis branch, which will be explored more fully in the next tutorial in this series. It allows you to generate predicted response(s) for any set of factors. To see how this works, click the Point Prediction node.


Point Prediction

You now see the predicted responses from this particular blend - the centroid. The Factors

Tool opens along with the point prediction window. Move the floating tool as needed by clicking and dragging the top border. You can also drag the handy red sliders on the component gauges to view other blends. Note that in a mixture you can only vary two of the three components independently. Can you find a combination that produces viscosity of 43? (Hint: push Urea up a bit.) Don’t try too hard, because in the next section of this tutorial you will make use of Design-Expert’s optimization features to accomplish this objective.

Explore a more precise way to specify component levels: Click the Sheet button to get a convenient entry form for

specific component values. Be careful though because the ingredients must add up to the fixed total you specified

earlier: 9 wt %. Design-Expert makes adjustments as you go — perhaps in ways you do not anticipate. Don’t worry: If

you get too far off, simply press Default to return to the centroid.

Analyze the Data for the Second Response

This last step is a BIG one. Analyze the data for the second response, turbidity (Y2). Be sure you find the appropriate polynomial to fit the data, examine the residuals, and plot the response surface. (Hint: The correct model is special cubic.)

Before you quit, do a File, Save to preserve your analysis. Design-Expert saves your models. To leave Design-Expert, use the File, Exit menu selection. The program warns you to save again if you’ve modified any files.

This tutorial gives you a good start using Design-Expert software for mixtures. We suggest you now go on to the Mixture Optimization Tutorial. You also may want to work the tutorials about using response surface methods (RSM) for process variables. To learn more about mixture design, attend Mixture Design for Optimal Formulations, a two-day workshop presented by Stat-Ease. Call or visit our web site at (www.statease.com) for a schedule.




Mixture Design Tutorial

(Part 2/2 – Optimization)

Introduction

This tutorial demonstrates the use of Design-Expert® software for optimization of mixture experiments. It’s based on the data from the preceding tutorial (Part 1 – The Basics). You should go back to that section if you’ve not already completed it. Much of what’s detailed in this Mixture Design Tutorial (Part 2 – Optimization) is a repeat of the Multifactor RSM Tutorial (Part 2 – Optimization). If you’ve already completed that RSM tutorial, simply skip over the areas in this tutorial that you find redundant.

Explore how to get an in-depth knowledge of optimization tools: For details about optimization, use the software’s

extensive on-screen program Help. Also, Stat-Ease provides in-depth training in its workshop titled Mixture Designs for

Optimal Formulations. Call for information on content and schedules, or better yet, visit our web site at

www.statease.com.

Start the program by finding and double clicking the Design-Expert software icon. To ensure being on the same page for this tutorial, go to File and Open Design file Mix-a.dxpx, which contains the experimental data as well as the response models.

The file you just loaded includes analyzed models as well as raw data for each response. Recall that the formulators chose a three-component simplex lattice design to study their detergent formulation. The components are water, alcohol, and urea. The experimenters held all other ingredients constant. They measured two responses: viscosity and turbidity. You will now optimize this mixture using their analyzed models.

Explore the modeling embedded in the data file: To see a description of the file contents, click the Summary node

under the Design branch at the left of your screen. Drag the left border and open the window to see the report better.

You can also re-size columns with your mouse. Now look at the bottom lines on responses R1 (viscosity) and R2

(turbidity). Observe under the “Model” column that these are fitted to quadratic and special cubic; respectively. That is

good to know.

P.S. For complete details on the models fitted, click the Coefficients Table node at the bottom left of the Design-Expert

screen (the last branch).

Numerical Optimization

Design-Expert software’s numerical optimization maximizes, minimizes, or targets:

A single response

A single response, subject to upper and/or lower boundaries on other responses

Combinations of two or more responses.

We lead you through the last above case: a multiple-response optimization. Under the Optimization branch of the program, click the Numerical node to start the process.


Setting the Optimization Criteria

Design-Expert allows you to set criteria for all variables, including components and propagation of error (POE). (We will get to POE later.) The limits for the responses default to the observed extremes.

Explore the options for setting goals: Now you reach the crucial phase of numerical optimization: assigning

“Optimization Parameters.” The program provides five possibilities for a “Goal” to construct desirability indices (di):

none (to disregard any given response), maximize, minimize, target, in range (simple constraint) and equal to

(components only).

Desirabilities range from zero to one for any given response. The program combines individual desirabilities into a

single number and then searches for the greatest overall desirability. A value of one represents the ideal case. A zero

indicates that one or more responses fall outside desirable limits. Design-Expert uses an optimization method developed

by Derringer and Suich, described by Myers, Montgomery and Anderson-Cook in Response Surface Methodology, 3rd

edition, John Wiley and Sons, New York, 2009.

In this case, components are allowed to range within their pre-established constraints, but be aware they can be set to desired goals. For example, because water is cheap, you could set its goal to maximize.

Options for goals on components

Notice that components can be set equal to specified levels. Leave water at its “in range” default and click the first response – Viscosity. Set its Goal to target-> of 43. Enter Limits as Lower of 39 and Upper of 48. Press Tab to set your entries.



Setting Target for first response of viscosity

These limits indicate it is most desirable to achieve the targeted value of 43, but values in the range of 39-48 are acceptable. Values outside that range have no (zero) desirability.

Now click the second response — Turbidity. Select its Goal to minimize, with Limits set at Lower of 800 and Upper of 900. Press Tab to set your entries. You must provide both these thresholds to get the desirability equation to work properly. By default they are set at the observed response range, in this case 323 to 1122. However, evidently in this case there’s no advantage to getting the detergent’s turbidity below 800 – it already appears as clear as can be to the consumer’s eye. On the other hand, when turbidity exceeds 900, it looks as bad as it gets.

Aiming for minimum on second response of turbidity

These settings create the following desirability functions:

1. Viscosity: if less than 39, desirability (di) equals zero from 39 to 43, di ramps up from zero to one


from 43 to 48, di ramps back down to zero if greater than 48, di equals zero.

2. Turbidity: if less than 800, di equals one from 800 to 900, di ramps down from one to zero if over 900, di equals zero.

Explore Screen Tips on numerical optimization: Do not forget that at your fingertips you will find advice about

using sophisticated features of Design-Expert software: Press the screen tips icon for an overview about Numerical

Optimization. Close out Tips by pressing X at the upper-right corner of its screen.

Changing Desirability Weights and the (Relative) Importance of Variables

The user can select additional parameters, called “weights,” for each response. Weights give added emphasis to upper or lower bounds, or emphasize a target value. With a weight of 1, di varies from 0 to 1 in linear fashion. Weights greater than 1 (maximum weight is 10) give more emphasis to goals. Weights less than 1 (minimum weight is 0.1) give less emphasis to goals. Weights can be quickly changed by ‘grabbing’ (via left mouse-click and drag) the handles (the squares ▫) on the desirability ramps. Try pulling the handle on the ramp down as shown below.

Weights change by grabbing handle with mouse

Notice that Weight now reads 10. You’ve made it much more desirable to get near the turbidity goal of 800. Before moving on, re-enter Upper Weights to its default value of 1 and press the Tab key. This straightens the desirability ramp.

“Importance” is a tool for changing relative priorities for achieving goals you establish for some or all of the variables. If you want to emphasize one variable over the rest, set its importance higher. Design-Expert offers five levels of importance ranging from 1 plus (+) to 5 pluses (+++++). For this study, leave Importance at +++, a medium setting. By leaving all importance criteria at their defaults, none of the goals is favored over any other.

Explore Help on numerical optimization: For an in-depth explanation of constructing desirability functions, and

formulas for weights and importance, select Help from the main menu. Then go to Contents and select Optimization,

then expand into the two Numerical Optimization topics — “How to” and “Statistical Details”. When you finish

viewing Help, close the screen by pressing X at the upper-right corner of its screen.

Running the optimization

Start the optimization by clicking the Solutions tab. Design-Expert brings up the Ramps view by default.



Ramps report on numerical optimization (Your results may differ)

The ramp display combines individual graphs for easier interpretation. The dot on each ramp reflects the factor setting or response prediction for that solution. The height of the dot shows how desirable it is. Press the different solution buttons (1, 2, 3,…) and watch the dots. The red ones representing the component levels move around quite a bit, but do the responses remain within their goals (desirability of 1)? Near the graph’s top, click the last solution (solution 13 in this case) on your screen. Does your solution look something like the one below?

Sub-optimum solution that ranks least desirable

If your search also uncovered the above local optimum, note that viscosity falls off target and turbidity becomes excessive, thus making it less desirable than the option for higher temperature.

Go back a step up on the Solutions Tool by pressing for the Report.


Numerical Optimization Report on Solutions (Your results may differ)

The report starts with a recap of your optimization specifications. Then it lists solutions in order of desirability. It ends with detailing of the starting points for the search.

Explore the Report: Scroll down this report to see how where the program starts its empirical searches for desirable

results. Multiple cycles improve the odds of finding multiple local optimums, some of which will be higher in

desirability than others. In this case Design-Expert grinds through 40 cycles of optimization, starting from the 10 design

points plus 30 more at random.

Go back now to the Solutions Tool and select the Bar Graph.

Solution to multiple-response optimization – desirability bar graph

The above bar graph shows how well each variable satisfies the criteria and the overall combined desirability: Values near one are good. This is not the best solution!

Optimization Graphs

Press the Graphs tab to view a contour graph of overall desirability. It now becomes obvious that at least somewhat desirable formulations fall with three distinct ‘sweet spots’ as indicated by the three graduated color areas within the blue background.



Desirability contour graph

The screen shot above came from a graph done showing graduated colors — cool blue for lower desirability and warm yellow for higher. Design-Expert software sets a flag at the optimal point for solution 13 (or whichever one is your worst).

Now click back through the numbered Solutions choices atop your screen until the flag relocates to the largest sweet spot (the one with the largest area) at the top of the triangular mixture space. To view the responses associated with this desirability (sweet spot), press the droplist arrow for Response and select Viscosity.

Most desirable point flagged (grid lines added — see sidebar to explore this)

Explore how to add grid lines: Right-click the graph and select Graph preferences, go to the Surface Graphs tab

and check on Show contour grid lines. There are many other options on this and other Graph preferences tabs. Look

them over if you like and then press OK to see how options specified by this tutorial affect your contour plot. If you like,

look at the optimal turbidity response as well.

P.S. For tutorial purposes, go back and press Default on all Graph Preference tabs to re-set the original layouts.

To view the desirability surface in three dimensions, again click Response and choose Desirability. Then from the floating Graphs Tool select 3D Surface.


3D view of desirability at default resolution in color

Now you can see one high ridge where desirability can be maintained at a maximum level over a range of compositions. Another high point can be achieved, but it requires sharp control of the composition. The other peak is less desirable (lower).

Explore going to very high graph resolution: So as not to tax user’s computers, Design-Expert defaults short of

maximum resolution. Try smoothing out the 3D desirability surface via a right-click over the graph, selecting Graph

Preferences and then on the Surface Graphs tab changing the Graph resolution to Very High. Press OK for the new

graph preferences. The go back and re-set things to the Default.

Adding Propagation of Error (POE) to the Optimization

If you have prior knowledge of the variation in your component amounts, this information can be fed into Design-Expert software. Then you can generate propagation of error (POE) plots showing how that error transmits to the response. Look for compositions that minimize transmitted variation, thus creating a formula that’s robust to slight variations in the measured amounts.

Start by clicking the Design node on the left side of the screen to get back to the design layout. Select Column Info Sheet from the floating Design Tool palette. Enter the following information into the Std. Dev. column: Water: 0.08, Alcohol: 0.06, Urea: 0.06, as shown on the screen below.

Column Info Sheet with standard deviations filled in



Now you can calculate propagation of error by generating graphs for each response. First, click the Viscosity analysis node and press the Model Graphs tab. Next, select View, Propagation of Error, which previously was grayed out. Also choose 3D Surface view. Now your screen should match what’s shown below.

3D view of the POE graph

The surface reaches a minimum where the least amount of error is transmitted (propagated) to the viscosity response. These minima occur at flat regions on model graphs where formulations are most robust to varying amounts of components.

Click the Turbidity node, press the Model Graphs button and select View, Propagation

of Error and look at its 3D Surface. Rotate it so you can see the surface best.

POE surface for turbidity

Now that you’ve found optimum conditions for the two responses, let’s go back and add criteria for the propagation of error. Click the Numerical optimization node. Select POE (Viscosity) and establish a Goal to minimize with Limits of Lower at 5 and Upper of 8.

A (5.000)B (2.000)

C (4.000)

90

100

110

120

130

140

150

PO

E(T

urb

idit

y)

A (3.000)

B (4.000)

C (2.000)


Set goal and limits for POE (Viscosity)

Select POE (Turbidity) and set its Goal also to minimize with Limits of Lower at 90 and Upper of 120.

Criteria for POE (Turbidity)

Now click the Solutions tab to generate new solutions with the additional criteria. (You may need to press Ramps on the Solutions Tool to get the view shown below.)



Solutions Generated with Added POE Criteria (Your results may differ)

The number 1 solution represents the formulation that best achieves the target value of 43 for viscosity and minimizes turbidity, while at the same time finds the spot with the minimum POE (most robust to slight variations in the component amounts).

Explore alternative solutions: If you can take the time, review the alternative solutions, which may be nearly as good

based on the criteria you entered. In this case, the number 2 solution, which you may or may not get due to the random

nature of the optimization, increases the water level (presumably cheaper) and reduces turbidity, so it may actually be

preferred by the formulators.

Viewing Trace Plots from Optimal Point

Continue on to the numerical optimization Graphs to look at the desirability contour plot. It does not look much different from before because adding POE criteria had only a small impact on the result. However, this is a good time to get a feel for the sensitivity of responses around the optimum point. Observe this by changing Response to Viscosity. Then select Trace from the Graphs Tool palette.


Trace plot viewed from optimal point

Now you see that changing component A (water) makes little difference on this response, whereas its very dependent on C (urea).

Take a look at the trace for the other response — turbidity. It looks even more interesting!


By shading out regions that fall outside of specified contours, you can identify desirable sweet spots for each response – windows of opportunity where all specifications can be met. In this case, response specifications are:

39 < Viscosity < 48 POE (Viscosity) < 8 Turbidity < 900 POE (Turbidity) < 120

To overlay plots of all these responses, click the Graphical optimization node. For the Viscosity response, if the following values are not already pre-set, enter a Lower limit of 39 and an Upper limit of 48.



Setting criteria for Graphical optimization: Viscosity response

Click the POE(Viscosity) response. If the following value is not already pre-set, enter an Upper limit of 8. Do not enter a lower limit — it will not be needed for the graphical optimization when simply minimizing.

Graphical criterion for POE of viscosity

Press forward to the Turbidity response and, if the following value is not already pre-set, enter an Upper limit of 900. This again is a minimization, so don’t enter a lower limit.


Setting criteria for turbidity

Click the POE(Turbidity) response and, if the following value is not already pre-set, enter an Upper limit of 120.

Press the Graphs tab to produce the “overlay” plot.

Graphical optimization

Notice that regions not meeting your specifications are grayed out, leaving (hopefully!) an operating window or “sweet spot.”

Notice the flag remains planted at the optimum. That’s handy! This Design-Expert display may not look as fancy as 3D desirability, but it is very useful to show windows of operability where requirements simultaneously meet critical properties. Grayed areas on the graphical optimization plot do not meet selection criteria. The clear “window” shows where you can set factors to satisfy requirements for both responses.

The lines that mark the high or low boundaries on the responses can be identified with a mouse-click. Notice that the contour and its label change color for easy identification. Click outside the graph to reset the contour and its label to the original color.

Let’s say someone wonders whether the 900 maximum for turbidity can be decreased. What will this do to the operating window? Find out by clicking the 900 turbidity contour



line – you know you’ve got it when it turns red. Then drag the contour until it reaches a value of approximately 750. Finally right-click over this contour, select Set contour value and enter 750.

Setting the turbidity contour value

Press OK to get the 750 contour level. Notice the smaller sweet spot has disappeared and the medium one considerably reduced in area. The flag no longer marks the right spot — this must be re-located via re-doing the numerical optimization. Do not do this. Instead go back to Criteria and reset Turbidity to an Upper limit of 900.

Explore adding uncertainty intervals around your window of operability: For Turbidity click Show Interval (one-

sided) and then press forward to Graphs. This pushes in the boundary by the confidence interval, thus accounting for

uncertainty in the mean prediction based on the model derived from this mixture experiment.

Confidence intervals (CI) superimposed on turbidity


If you are subject to FDA regulation and participate in their quality by design (QBD) initiative, the CI-bounded window

provides a relatively safe operating region — a functional design space — for any particular unit operation.

Manufacturing design space requires tolerance intervals. This tutorial experiment provided too few runs to support

adding the CI for viscosity, much less the imposition of TIs. Learn how to size designs properly for manufacturing QBD

by attending our Designed Experiments for Pharma workshop.

Graphical optimization works great for three components, but as the number increases, it becomes more and more tedious. Once you find solutions much more quickly by using the numerical optimization feature, return to the graphical optimization and produce outputs for presentation purposes.

Response Prediction at the Optimum

Click the Confirmation node (near bottom left on your screen). Notice it defaults to your first solution.

Confirmation set to Solutions 1 (yours may be different)

This defaults to the prediction interval (PI) for a single point.

Explore the Confirmation feature: You had best replicate the optimal formulation six or so times and then click the

Enter Data option to type these into Design-Expert. It then computes the Data Mean and puts this in the middle of the PI

values for evaluation. Try entering some numbers for yourself and see what happens.


Now that you’ve invested all this time into setting up the optimization for this design, it is wise to save your work. Click the File menu item and select Save As. You can now specify the File name (we suggest tut-MIX-opt) to Save as type *.dxpx” in the Data folder for Design-Expert (or wherever you want to Save in).

Final Comments

We feel that numerical optimization provides powerful insights when combined with graphical analysis. Numerical optimization becomes essential when investigating many components with many responses. However, computerized optimization does not work very well in the absence of subject-matter knowledge. For example, a naive user may define impossible optimization criteria. The result will be zero desirability everywhere! To avoid this, try setting broad, acceptable ranges. Narrow down the ranges as you gain knowledge about how changing factor levels affect responses. Often, you will need to make more than



one pass to find the “best” factor levels that satisfy constraints on several responses simultaneously.

Using Design-Expert software allows you to explore the impact of changing multiple components on multiple responses — and to find maximally desirable solutions quickly via numerical optimization. For your final report, finish up with a graphical overlay plot at the optimum “slice.” (Don’t forget you can set goals on the components themselves. For example, in this case it might be wise to try maximizing the amount of cheap water.)

Learn more about mixture design methods at our workshop titled Mixture Designs for Optimal Formulations. To get the latest class schedule, go to the Training link at www.statease.com. Also, we appreciate your questions and comments on Design-Expert software. Address these to [email protected].

Postscript: Adding a Cost Equation

In the comments above, we suggested you consider maximizing the cheapest ingredient – water in this case. Conversely, you may have an incredibly expensive material in your formulation that obviously needs to be minimized. With only a small amount of effort, you can set up cost as a response to be included in Design-Expert’s numerical optimization.

Re-open the Mix-a.dxpx file. In the Design branch, right-click the last response column. From the menu, select Insert Response, After This Column.

Inserting a new response

Next, right-click the new untitled response header and select Simulate. Then choose Use

equation in analysis.

Preparing to enter cost equation

Press Next and enter into the area provided .5b+.2c (alcohol at $0.50 per kilo and urea at $0.20 cents – assume water costs practically nothing).




Entering the cost equation

Press Finish to accept the equation and calculate costs for all formulations in this mixture design. To make these more presentable, right-click the Response 3 column header, select Edit Info, and in the dialog box enter Response Name as Cost and Response units in $/kg. Also change Format to 0.0. Press OK.

Costs calculated and formatted

Now, under the Analysis branch, click the Cost node to bring up the model graph directly — no modeling is necessary because you already entered the deterministic equation.

Contour plot of cost



The water shows blue due to it being so cheap.

This sets the stage to include cost in your multiple response optimization. As pictured below, go to Optimization node Numerical, select Cost and set its Goal to minimize.

Minimizing cost

Pressing Solutions at this stage only tells you what you already know: The lowest cost formula is at the greatest amount of water within the specified constraints. Re-enter the goals for viscosity and turbidity and their POEs if you like, but it really isn’t necessary now. Wait until you do your own mixture design and then make use of this postscript tip to take costs into account.

DX9-06-2-Combined.docx Rev. 5/2/14

Design-Expert 9 User’s Guide Combined Mixture-Process Tutorial 1

Combined Mixture-Process Tutorial

Introduction

Design-Expert® software offers designs that combine mixture components and process factors (numeric and/or categoric). In this tutorial, you will apply Design-Expert’s unique features for mixture-process design and analysis by investigating John Cornell’s famous fish-patty experiment from his textbook Experiments with Mixtures, 3rd edition, published by John Wiley and Sons, New York. Zip through it all by skipping the sidebars, or, if you like to explore, take the time to study these.

Explore the basics of mixture design: Before embarking on this tutorial, you should first complete the previous

tutorial on mixture design.

P.S. Stat-Ease provides in-depth training on combined mixture-process in its Advanced Formulation. Call for

information on content and schedules, or better yet, visit our web site at www.statease.com.

The food formulators hope to make something delicious from a blend of three tasty-sounding (?) fish: mullet, sheepshead, and croaker. Yum! The seven blends in the design include one each of the three fishes, plus three binary blends, and a blend of one-third each of the individual types of fish. The fish patties can be cooked at varying deep-frying time, oven temperature, and oven time. Combining each of these process factors at two levels for all seven blends creates a total of 56 experimental runs. The diagram below shows the blends as points on triangles, repeated at each of the eight corners of a cube representing the process.

Fish-patty design: Seven blends (on triangles) at eight process combinations (cube)

The response measure is patty texture. (Never mind about taste!)

The Experiment as Originally Run

Start the program by finding and double clicking the Design-Expert software icon. Take the quickest route to initiating a new design by clicking the New Design option on our opening screen as shown below.

2 Combined Mixture-Process Tutorial Design-Expert 9 User’s Guide

Opening screen – New Design option highlighted in red

Click the Combined folder tab and select the User Defined option. The user-defined option allows you to reproduce Cornell’s design with all points chosen. Normally, to save on runs, you want to use the first choice on the list, Optimal, which selects an ideal subset of design points from a candidate set for a specified model. We’ll explore this option later.

For Mixture 1 Components, select 3 from the droplist. (Notice that Design-Expert offers the option for adding a second mixture, for example in a two-layer cake, film, or coating. For the number of Numeric Factors choose 3. Leave Categoric Factors at zero, but keep in mind for future experimentation that you can add discrete variables – such as who supplies a given component.

Specifying the combined user-defined design

Press Continue and enter the Total for mixture components (fish types) at 100 and Units as %. Then enter fish names as shown below. Low limits remain zero. Set all high limits to 100.

Entering mixture details

Press Continue to move on to the process design factors. Enter factor names, units, and ranges (low – L[1] and high – L[2]) as shown below.



Entering process details

Press Continue to define the model you want to design for the combination of mixture and process variables in this experiment. The default model is quadratic by quadratic. Click the Edit model button and change the default for Process Order to Linear.

Specifying the design model for the mixture and process variables

Click the OK button to implement the change. Now, consider your “candidate” points. Normally only some of these points would actually make it to the design, but by choosing the User Defined option, you get all of them. However, there’s a problem—the Mixture candidate set of points (Mixture tab at upper-left screen box) contains more points than those chosen by Cornell. Fix this by clicking off Axial check blends, and Interior check

blends. Only vertices, centers of edges, and overall centroid should now be checked. Press Calculate points to get a count — this subtotals to 7 for the mixture candidate points.

Candidate points for mixture

Now click the Process candidate set of points tab (right of Mixture tab). This design also contains more points than those chosen by Cornell, so click off every checkmark except Vertices. Press Calculate points to update total run count.


Candidate points for the process

You should now see a subtotal of 8 process combinations, which combined with the 7 mixture candidates, generates 56 total runs (points).

Press Continue to see the response specification form. Enter Ave Texture with units of gm*10^-3.

Entering response name and units

Press Finish to complete the design. Design-Expert now displays the 56 experiments in random run order.

Analyze the Results

To view response data, right-click the Response column header and select Simulate.

Simulating the response

This brings up a dialog box defaulted to Load an existing simulation. Accept this by pressing Finish. Then find and double-click the Fish Patties.simx to open it and run the simulation.

Look over all resulting response data: Results within the range of 2 to 3.5 are desirable. To preserve it, go to File and select Save As. Change the file name to FishPatties.dxpx or some other unique name of your choice.

To analyze the data, click the Analysis node labeled Ave Texture. Then click the Fit

Summary tab. Next, on the floating Bookmarks press forward to Sum of Squares. You



now see a unique matrix of probabilities that helps you determine the best crossed model for mixture and process. As shown below, the software provides suggestion(s) on which combination is best—in this case, a quadratic (“Q”) mixture model crossed with a linear (“L”) process model.

Suggested model — quadratic (mixture) by linear (process)

Explore details on how to interpret the statistics for combined mixture-process models: For a detailed explanation

about how to interpret sequential p-values in this Fit Summary table for combined designs, see Stat-Ease’s Handbook for

Experimenters (provided free of charge to all registered users of Design-Expert software) section titled “Combined

Mixture/Process Analysis Guide.” This section is available upon request to trial users.

Now click the Model tab atop your screen. Design-Expert uses the suggested model as its default. Accept this for the moment by clicking ahead to the ANOVA. Notice that many of the model terms, particularly ones of third-order (for example, ABF) exhibit high Prob>F values. This is a byproduct of crossing the two mixture and process models, which creates many superfluous higher-order terms. When analyzing designs like this, you will often find it beneficial to reduce your model to only the significant terms, subject to requirements for maintaining hierarchy (no interactions without their parent terms).


Initial ANOVA (no model reduction)

To reduce the model, go back to Model. You could manually deselect the insignificant terms observed from the ANOVA, but it is quicker to let Design-Expert do this for you. Change selection mode to Backward.

Selecting backward reduction

This enables automatic model reduction by a backward (stepwise) algorithm. The criterion for elimination is a probability value of 0.1000, as specified in the Alpha Out field. Click the ANOVA. As shown below, the software shows you a report on eliminated terms.



Report showing removed terms via backward model reduction

The ANOVA table shows that the reduced model produces a highly-significant F Value (p-value < 0.0001). On the floating Bookmark press ahead to the R-Squared statistics — they look really good.

Analysis of Variance (partially shown) for Crossed Model

Click the Diagnostics tab and examine the graphs of residuals.


Normal plot of residuals — looks good

The normal plot of residuals looks good, so move on to the Model Graphs to view the response in mixture (triangular) space.

Model Graph with view of contour plot for mixture portion of combined design

Click or grab various process factors on the floating tool bar and adjust them to see how response changes. For example, push all three bars to the right as shown below.



Process slide-bars pushed to their maximum levels

Notice by the hotter colors that texture increases at the 100 percent mullet (top of triangle).

Now right-click the Factors Tool where it shows D:oven temp and select X2 axis. As shown below, the view changes to process (rectangular) space.

Changing to process view

Now shift back to mixture coordinates by right-clicking the Factors Tool where it lists A:mullet. Select X1 axis.


Back to mixture space

On the Graphs Tool click 3D Surface and notice the strong tilt in the A-B axis.

3D mixture view

Press ahead to the 3D Surface Mix-Process display. It’s very likely you have never seen a plot like this!

3D plot of two mixture components versus one process factor!



Now you see the impact of shifting component A (mullet) in direct substitution for B (sheepshead) on one axis versus the process factor D (oven temp) on the other axis. Isn’t that something!

Continue if you like and explore different combinations of the mixture components and process factors in 2D contour view and 3D plots. Keep in mind that any textures outside of 2 to 3.5 are unacceptable.

Find the Optimal Solution

The goal of the experimental program is to learn how to produce fish patties with a texture in the range of 2.00 to 3.50 x 103 grams. The target is 2.75. To find optimal combinations of formulas and processing, click the optimization node labeled Numerical. Then select Ave

Texture. Select a Goal of target-> and enter 2.75. For Limits enter a Lower of 2 and Upper of 3.5. Leave the weights and importance settings at their default levels. Your screen should now look like that below.

Optimization criteria for texture

Click the Solutions tab. If not already there by default, mouse to the Solutions Tool and select Ramps. Notice the many solutions listed atop your screen. (Due to the random nature of the optimization algorithm, your screen may show fewer or more solutions.) The software defaults to the most desirable solution.


One of many solutions for optimized texture (your results may vary)

Click solution number 2 and beyond: You will see many combinations that satisfy the texture requirements. Next press Graphs and select from the Graphs Tool the 3D

Surface. Move your mouse cursor over the graph and when it turns into a hand () grab it by depressing the left mouse button and rotate it to a vantage point like that shown below.

3D desirability view (your results may vary)

Explore other views: Check out the desirability surface for other solutions. Remember that by right-clicking any of

the process factors on the Factors Tool, you can create a graph in process space.

To preserve your modeling and optimization work, select File, Save or press the icon.

The Experiment via Optimal Design

As noted in the introduction, we wanted to reproduce the fish patty case exactly as reported by Cornell in his textbook Experiments with Mixtures. Now let’s try an optimal design, which is much more efficient. We recommend you use this approach when designing your own mixture-process experiment. Assuming you are continuing from before, proceed by



choosing File, New Design. The software asks whether to “Use previous design info?” Click Yes.

Re-using previous design information

You see the builder from the previous user-defined design session. Select Optimal

(custom).

Optimal design option for combined mixture-process design

Press Continue to see that all component names and Low/High values remain filled in for you. Press Continue again to see your process factors, again at their previous ranges. Press Continue one more time to bring up the Optimal Design screen. Notice it’s already set up for the quadratic-linear base model that you want for the fish-patties process-mixture design.

The objective of this exercise is to pick a subset of runs from Cornell’s actual experiment so the search must be limited to specific combinations – not just any coordinate within the mixture-process space. Therefore you must change the Search droplist from its defaults or “Best” to Point Exchange, as shown below.

Optimal design search by Point Exchange


Leave the optimality criterion at its default of “I” — a good choice in this case where continuous variables must be optimized. This criterion will dictate the selection of the minimal subset of model points from the candidate set, which in a moment you will specify as the original combinations run by Cornell.

Explore other views: Now is a good time to view Help on the Search term “Candidate Point Creation”. From there

you can link to details on the I-optimal and alternative algorithms offered by Design-Expert. Exit out of Help (X) to get

back to the work of building an optimal design for fish patty mixture and process development.

Because the original data had no replicates, set Replicates to 0. Hit the tab key and you see a total of 29 runs. Leave “Model points” and “To estimate lack of fit” at their default levels.

Default for replicates reduced to zero to match textbook case

Before continuing, you need to re-create the points originally selected by Cornell. (Recall doing this earlier when you set up the User Defined design.) Click Edit candidate points. In the Mixture tab, check off Thirds of edges, Triple blends, Axial check blends, and Interior check blends. Press Calculate points. You now see a subtotal of 7 mixture points comprised of vertices (3), centers of edges (3) and the overall centroid (1). Go back to the screen shot shown on this earlier to be sure you have it right!

Next, press the Process tab and check off all points except for vertices. Click Calculate

points. This produces a subtotal of 8 process points. You now have 56 total candidate points, the same as earlier when you set up this case study as a User Defined design.

Click OK to move on to the response specification. Accept the default response name and units (as in the user defined example) by pressing Continue.

Design-Expert now builds the optimal design. Due to random elements in the algorithm (configurable via Options), the 29 runs chosen may differ each time you do an optimal design, but they will be essentially identical in their matrix attributes.

Fish patty design: one possible subset of 29 I-optimal combinations

Remember that the point of this follow up to the first part of the tutorial is to show how you can get by with only a fraction of all possible mixture-process combinations by using an optimal design — in this case one that includes combinations from within the original experiment. Thus, this is a “what-if” exercise. If you were to build a completely new design for this case, it would be advisable to include the 5 replicates that Design-Expert software



suggests by default. This would provide 4 degrees of freedom to estimate pure error, which would allow a test for lack of fit, and generally improve the quality of the design.

Final comments

In all but the simplest cases, a User Defined design (all possible combinations of mixture and process points) is too large to be practical. Consider using an optimal design, which provides a reasonable subset of points to estimate the model you specify. For any combined design, but especially if done via optimal for a minimum-run experiment, it’s advisable to over-ride any suggested model that falls short of what can be estimated without aliases. Then perform a backward elimination to rid the model of the large number of superfluous terms that are characteristic of these experiments that combine variables from the mixture and the process. If all goes well, you can blend your fish and bake it too. However, we have a suggestion: Include taste as one of your sensory attributes (not just texture!).

Design expert 9 tutorials 2015

Science

Transcript of Design expert 9 tutorials 2015