Attack the Variance.hurst-1
-
Upload
conjugateddiene -
Category
Documents
-
view
215 -
download
0
Transcript of Attack the Variance.hurst-1
-
7/29/2019 Attack the Variance.hurst-1
1/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 1
Attack the Variance, Course 1Tools to Understand Variance in Analytical
Methods
Olivier Guise
Ph.D. Chemist
Roger Hurst
Ph.D. Chemist
Pittcon Short CourseMarch 19, 2013
-
7/29/2019 Attack the Variance.hurst-1
2/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 2
Section 1 Understanding your data
Evaluating data quality are there potential problems? Concepts: Mean, standard deviation, precision, accuracy, Z-score Tools: Hypothesis tests, p-values, Analysis Of Variance (ANOVA), t-
tests, outlier tests, regression analysis
Section 2 How good is the ruler?
Where is the variation sample or measurement or both? Concepts: measured vs. observed variation, effect of interactions,how to correct variation
Tools: Gage Repeatability and Reproducibility (GRR), fishbonediagram (cause and effect diagram)
Section 3 Identify sources of variation . . . Then optimize! Which factors are really important to optimize? Concepts: Knowing which knobs to turn, which way to turn them,
and how much in order to reach the optimal method. Tools: Screening Design of Experiment(DOE) & optimization DOE
Pittcon March 2013
Course Overview
-
7/29/2019 Attack the Variance.hurst-1
3/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 3
Your Names & Backgrounds
-
7/29/2019 Attack the Variance.hurst-1
4/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 4
Students Course Expectations
-
7/29/2019 Attack the Variance.hurst-1
5/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 5
Be involved!
Think constantly about howyou are going to use thiswhen you get back
Ask lots of questions!
Participate in the exercises they are designed to makesure you go home with morethan a notebook!
Cell phone courtesy
Bring up your casestudies that you would likehelp with, from us or fromthe group
If you get ahead, pleasehelp your neighbor. Weneed to stay together toget through as muchmaterial as possible, asquickly as possible.
Instructors Course Expectations
-
7/29/2019 Attack the Variance.hurst-1
6/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 6
Pittcon March 2013
Section 1
Understanding Your Data
SABIC Innovative Plastics
Analytical Technology
Evaluating data quality are there potential problems? Concepts: Mean, standard deviation, precision, accuracy, Z-score Tools: Hypothesis tests, p-values, Analysis of Variance (ANOVA),
t-tests, outlier tests, regression analysis
-
7/29/2019 Attack the Variance.hurst-1
7/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 7
21
1
2
1
)(
n
xx
s
n
i
iMean
1 2 3-1-2-3 1 68.20 % of all data 2 95.54 % of all data 3 99.7% of all data
Process Variability -multiple results fromthe same process
NormalDistribution
(Gaussian)
-4-5-6 4 5 6
6 99.99999980% of all data
Standard Deviation
-
7/29/2019 Attack the Variance.hurst-1
8/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 8
Accuratenot precise
Addressvariation
issues
Target AnalogyCorrective Action:
Precise,Not accurate
Calibration tocorrect bias
True value
Precise andaccurate
Ideal
Precision vs. Accuracy
True value
True value
-
7/29/2019 Attack the Variance.hurst-1
9/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 9
PoorProcess
Capability
MeasurementProcess
LSL USL
Target
LSL USL
Target
ExcellentProcess
Capability
LSL USL
Target
So HOW good is good enough??
MeasurementProcess
MeasurementProcess
-
7/29/2019 Attack the Variance.hurst-1
10/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 10
MeanProcessVariability -
multiple resultsfrom the same
process
When you have aspecified tolerance:
If the limits fall on thevertical lines, the Z
score is . . .
1 2 3-1-2-3 -4-5-6 4 5 6
Z = 1
Z = 2
Z = 3
Z = 4
Higher Z isbetter!
Z Score: A Measure of Variance Relative to Tolerance
-
7/29/2019 Attack the Variance.hurst-1
11/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 11
Z =USL - x
sobserved
Z =12.0 11.0
0.4
= 2.5
This methodis 2.5 Sigma.
Z = 12.0 7.02.0
= 2.5
This methodis 2.5 Sigma.
LSL USL
x =7.0sobs = 0.4
Z =12.0 7.0
0.4
= 12.5
This methodis 12.5 Sigma.
(Use spec limit nearer to themean . . . may be LSL)
Process
LSL USL
Target
x =11.0sobs = 0.4
12.02.0
7.0
LSL USL
x =7.0sobs = 2.0
Is it Good Enough? . . . Estimate Z Score
-
7/29/2019 Attack the Variance.hurst-1
12/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 12
Minitab Graphical Summary:A good way to get an overall feel for your data
-
7/29/2019 Attack the Variance.hurst-1
13/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 13
Minitab Descriptive Statistics:A good way to get an overall feel for your data
54.052.551.049.548.046.545.0
Median
Mean
50.049.549.048.548.047.547.0
1st Quartile 46.958
Median 47.930
3rd Quartile 49.852
Maximum 53.570
46.878 50.238
46.905 49.972
1.616 4.288
A -Squared 0.31
P-V alue 0.496
Mean 48.558
StDev 2.349
V ariance 5.517
S kewness 1.00719
Kurtosis 1.26643
N 10Minimum 45.390
A nderson-Darling Normality Test
95% C onfidence Interval for Mean
95% C onfidence Interv al for Median
95% C onfidence Interval for StDev
95% Confidence Intervals
Summary for Masses
-
7/29/2019 Attack the Variance.hurst-1
14/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 14
Ho = the null hypothesis (no difference)
Ha = the alternative hypothesis (difference)
True State Test Conclusion
Not Guilty Guilty
Not Guilty Acquit Send to Jail
Guilty Acquit Send to Jail
True State Test Conclusion
In Spec Out of Spec
In Spec Pass Fail
Out of Spec Pass Fail
Which Error is worse?
consider the liability
Compare samplemean and
population of data,two sample means,two sample std.deviations, etc.
Hypothesis Testing
-
7/29/2019 Attack the Variance.hurst-1
15/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 15
Hypothesis Testing (t-test used to compare means):
Ho: Xbar-A= Xbar-B Ho: Xbar-B = Xbar-C Ho: Xbar-A = Xbar-C
Ha: Xbar-A Xbar-B Ha: Xbar-B Xbar-C Ha: Xbar-A Xbar-C
Stech #1= 3 Stech#2= 1
How do we determine if these are different?
Technique #2Technique #1
A B CA B C
-
7/29/2019 Attack the Variance.hurst-1
16/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 16
t-distribution (Student t)
Degrees of
Freedom (df)
0.10 0.05 0.005
1 3.078 6.314 63.657
5 1.476 2.015 4.032
10 1.372 1.812 3.169
tcritical
= 0.05
Step 1. Find tcritical
df = n-1
Where nis # ofsamples
T-critical: Determining if two means are different
-
7/29/2019 Attack the Variance.hurst-1
17/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 17
t-distribution (Student t)
tcriticaln
s
xtcalc
error
If tcalc > tcritical reject Ho, accept Ha
There is a difference between means
Ho: Xbar-A= Xbar-B Ha: Xbar-A Xbar-B
Step 2. Determine tcalc
tcalc
BUTwe cannot run t-tests every time we doan analysis (too many factors!!!), SO.
T-Calculated: Determining if two means are different
-
7/29/2019 Attack the Variance.hurst-1
18/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 18
Hypothesis Testing
Breakout #1A
Data File: T-test F-test.xls
-
7/29/2019 Attack the Variance.hurst-1
19/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 19
Instructions:
1. Open the file, T-test F-test.xls
2. Copy and Paste into Minitab.
3. Stack Columns to make one column.
4. Determine if there is a difference in mean value of the groups
First, test for normality
Use Homogeneity of Variance to compare the variance
Use t-test to compare the means
Chemicoolest, Inc. has developed a new mass flow-meter for
industrial process control. Two operators have run experimentsunder the same settings to test the system and operability. The mass
data are stored in the data file, T-test F-test.xls. Use Minitab to
asses the normality, variance, and mean response for this
experiment.
Breakout #1A
-
7/29/2019 Attack the Variance.hurst-1
20/126Slide-Attack the Variance, Course 1 (Pittcon 2013) 20
Stack columns
Breakout #1A Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
21/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 21
Check Normality for Each Operator
(or Subset) Separately
Breakout #1A Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
22/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 22
(Normal if p > 0.05)
Accept Null Hypothesis: Normal
Breakout #1A Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
23/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 23
Test for Homogeneity of Variance
Breakout #1A Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
24/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 24
Reject Null Hypothesis: Unequal Variances
F-test is fornormal data
Levenes test is
for non-normaldata
Breakout #1A Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
25/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 25
2-sample t-test compares themeans of 2 distributions
Do not assume equal variance.F-test showed unequal variance.
Breakout #1A Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
26/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 26
2-sample t-test compares themeans of 2 distributions
Breakout #1A Screen Shots
Accept Null Hypothesis: Means Appear Equal
-
7/29/2019 Attack the Variance.hurst-1
27/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 27
Hypothesis Testing
Breakout #1B
Data File: Hypothesis Testing Class Example.xls
-
7/29/2019 Attack the Variance.hurst-1
28/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 28
Instructions:
1. Open the file, Breakout #1B Class Example.xls
2. Copy and Paste into Minitab.
3. Stack Columns to make one column.
4. Determine if there is a difference in mean value of the groups First, test for normality
Use Homogeneity of Variance to compare the variance
Use t-test to compare the means
You just developed a new FTIR method for measuring co-polymer
composition. You and another colleague decide to gauge the methodby running some replicates and comparing the results. The mass
data are stored in the data file, Breakout #1B Class Example.xls.
Use Minitab to asses the normality, variance, and mean response for
this experiment.
Breakout #1B
-
7/29/2019 Attack the Variance.hurst-1
29/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 29
Breakout #1B Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
30/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 30
Need to investigate more . . .
p > 0.05, normal p < 0.05, Uh-oh
Breakout #1B Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
31/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 31
All data are normal when
analyzed together.
What can cause this?
p > 0.05, normal
All data, bothoperators
Descriptive statistics shows
a possible outlier.
What would you do?
Breakout #1B Screen Shots
54.052.551.049.548.046.545.0
Median
Mean
50.049.549.048.548.047.547.0
1st Quart il e 46.958
Median 47.930
3rd Quart ile 49.852
M aximum 53.570
46.878 50.238
46.905 49.972
1.616 4.288
A -Squared 0.31
P-V alue 0.496
Mean 48.558
StDev 2.349
V ariance 5.517
S ke wne ss 1. 00719
K urtosis 1.26643
N 10
Minimum 45.390
A nderson-Darling Norm ality Test
95% C onfidence Interval for Mean
95% C onfidence Interval for Median
95% C onfidence Interval for StDev
95% Confidence Intervals
Summary for Masses
-
7/29/2019 Attack the Variance.hurst-1
32/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 32
Looking at Levenes Test (since
we had one non-normal dataset), we see that the variances
are not statistically different.
Breakout #1B Screen Shots
Operator B
Operator A
108642
Operators
95% Bonferroni Confidence Intervals for StDevs
Operator B
Operator A
54.052.551.049.548.046.545.0
Operators
Masses
Test S tatistic 0.71
P-Value 0.746
Test S tatistic 0.06
P-Value 0.818
F-Test
Levene's Test
Test for Equal Variances for Masses
Given the data, we could
accept the null
hypothesis if the data
were normal. . . . Howcan we be more
certain??
-
7/29/2019 Attack the Variance.hurst-1
33/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 33
Multiple measurements (n>3) were taken on a given sample. One appearsto deviate wildly, hence could significantly influence the mean, stdev, and
distribution. Can I remove this point with 90% confidence?
30.38
30.23
30.34
29.98
30.29 30.31
29.7
29.8
29.9
30
30.1
30.2
30.330.4
30.5
1 2 3 4 5 6
Measurement No.
Content(%)
Apply a Grubbs Test:
t.v. =|29.98 30.26|
0.1438= 1.947
Compare t.v. to test Grubbs table
t.v. =(Grubbs
test value)
|xi x|
stdev
Outlier Tests When can I throw out a bad result?
-
7/29/2019 Attack the Variance.hurst-1
34/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 34
In this case, n = 6 for a one-
tailed distribution.
G(6, 90) = 1.729
Since 1.947 > 1.729
the measurement is anoutlier.
P(one-tailed)
P(two-tailed)
Grubbs Table
-
7/29/2019 Attack the Variance.hurst-1
35/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 35
Words of caution with outlier tests:
Apply to multiple measurements of the same
sample, not single measurements of a set of
samples.
First, try to identify and resolve the cause of the
outlier. If you have an outlier and need to report a
single value to represent the result, use the
median value, rather than the mean.
Qobs =Suspect value nearest value
Range
If Qobs > Qcrit, then you
can reject it.
Q-Test Another type of outlier test
-
7/29/2019 Attack the Variance.hurst-1
36/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 36
Example:
A set of samples were made
spanning a range of different
compositions.
Each sample was measuredonce.
The measured values were
plotted vs. the theoretical
values (formulation)
Is this due to measurement variation or a bad sample?
Can it be thrown out? . . .
Use a standardized (studentized) residual test.
Validation Set: Measured vs.
Theoretical
y = 0.9942x + 0.1922
R2
= 0.9468
18.0
18.5
19.0
19.520.0
20.5
21.0
21.5
22.0
18.0 19.0 20.0 21.0 22.0
Theoretical Content (%)
MeasuredC
ontent(%)
|yactual yfit|
= residual
Standardized Residuals How to treat outliers in aset of regression data
-
7/29/2019 Attack the Variance.hurst-1
37/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 37
Strategy:
Use Minitab to plot the
standardized residuals.
Any residual > 3 is outside the
99.7% confidence interval and
can be considered an outlier.
Investigate and resolve the
reason for the outlier if
necessary.
Class Exercise:
standardized residuals.mpj
Standardized Residuals contd
-
7/29/2019 Attack the Variance.hurst-1
38/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 38
2624222018161412108642
4
3
2
1
0
-1
-2
Observation Order
StandardizedResidual
Versus Order(response is Measured)
well over 3 . . .outlier
Tip - Its still recommended to resolve why the sample wasbad than to simply reject it and forget about it.
Standardized Residuals contd
-
7/29/2019 Attack the Variance.hurst-1
39/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 39
This tool will determine if the response means associated with the groups arefrom the same population. (Used when data sets vary only in one factor).
Ho: m1 = m2 = m3 = m4
Ha: At least one m different from the others
If p > 0.05 then accept Ho
If p < 0.05 then
One of the means is different than the others.
One-Way ANOVA: Used for Single-Factor Studies
-
7/29/2019 Attack the Variance.hurst-1
40/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 40
Grand mean
diameter
Measuremultiple
diameters for
each circle.(Distributionresults from
measurementerror may be
instrument and/or
technique.)
The differenceWITHIN each circleis small relative to
the differenceBETWEEN circles.
. . . So one circle isa differentdiameter.
This doesnt require
ANOVA . . . But whatif we add variation to
the groups?
Example: A single measurement (diameter) and a single factor (circle #).
One-Way ANOVA: Used for Single-Factor Studies
-
7/29/2019 Attack the Variance.hurst-1
41/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 41
Measuremultiple
diameters foreach shape.
(Distribution ofresults comes
frommeasurement
error and actual
changes indiameter.)
Grand mean
diameter
Now it
helps tohaveANOVA
One-Way ANOVA: Used for Single-Factor Studies
-
7/29/2019 Attack the Variance.hurst-1
42/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 42
One-way Analysis of Variance
Source DF SS MS F P
Variety 5 2.43507 0.48701 56.22 0.000
Error 18 0.15593 0.00866
Total 23 2.59100
Tabular Evaluation of 1-way ANOVA OutputAccept Ha: Factor Is Significant
TSS = SSW + SSBTotal Sum of Squared Differences
Sum of Squares Within Group
Sum of Squares Between Group
One-Way ANOVA: Interpretation of Minitab Results
ANOVA: a method ofT W ANOVA G hi ll
-
7/29/2019 Attack the Variance.hurst-1
43/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 43
Factor 2
Sample 2
SSB1 = Sabc (for all levels of factor 1)SSB2 = the same for all levels of factor 2
SSE is the total sum of squares error
Fact
or1
Oper-ator 1
2,1
1,21,1
2,2
3,1 3,2
TSS=SSB1+SSB2+SSE
Ybar overall mean
ANOVA: a method ofdetermining the differences inmeans based on the variationin measuring those means.Used when 2 factors are varied:
Such as operator and sample
Oper-ator 2
Sample 1
Sample 2
Sample 3
Sample 3
Avg of 3,1 & 3,2
c
Sample 1
Avg of 1,1 & 1,2a
Avg of 2,1 & 2,2
b
Two-Way ANOVA: Graphically
-
7/29/2019 Attack the Variance.hurst-1
44/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 44
ANOVA
Breakout #2
Data Files:
One-Way ANOVA.xls
Two-Way ANOVA.xls
-
7/29/2019 Attack the Variance.hurst-1
45/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 45
Breakout 2A: One-Way ANOVA
Open data tab 2A in Breakouts_1.xls
Copy/paste from Excel to Minitab
Stack Columns into one column
Analyze One-Way ANOVA
What can you conclude about the means?
Breakout 2B: Two-Way ANOVA
Open data tab 2B in Breakouts_1.xls
Copy/paste from Excel to Minitab
Stack Columns into one column
Analyze Two-Way ANOVA
Are the means the same in terms of Operator? In terms ofSample?
Breakout #2: One-Way ANOVA and Two-WAY ANOVA
-
7/29/2019 Attack the Variance.hurst-1
46/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 46
1. Copy/paste from Excel2. Analyze ANOVA
3. Choose Plots
Breakout #2A One-Way ANOVA Screen Shots
Breakout #2A: One Way ANOVA Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
47/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 47
Mean Square (MS)
(Sum of Squares/df)
F value
(MSFactor/MSError)
p is probability of an
alpha error; NOTsignificant if p>0.05
Reject NullHypothesis:
Unequal Means
Breakout #2A: One-Way ANOVA Screen Shots
Operator 3Operator 2Operator 1
34.850
34.825
34.800
34.775
34.750
Data
Boxplot of Operator 1, Operator 2, Operator 3
Breakout #2B: Two Way ANOVA Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
48/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 48
1. Copy/paste from Excel
2. Stack Columns into one column
3. Analyze ANOVA
Breakout #2B: Two-Way ANOVA Screen Shots
Breakout #2B: Two Way ANOVA Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
49/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 49
Samples are differentOperators are the same
Breakout #2B: Two-Way ANOVA Screen Shots
Operators
sample
Percent Acid Operator 2Percent Acid Operator 1
43214321
2.00
1.75
1.50
1.25
1.00
0.75
0.50
Data
Individual Value Plot of Data vs Operators, sample
-
7/29/2019 Attack the Variance.hurst-1
50/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 50
Pittcon March 2013
Section 2
How good is the ruler?SABIC Innovative Plastics
Analytical Technology
Where is the variation sample or measurement or both? Concepts: measured vs. observed variation, effect of
interactions, how to correct variation Tools: Gage Repeatability and Reproducibility (GRR),
fishbone diagram (cause and effect diagram)
-
7/29/2019 Attack the Variance.hurst-1
51/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 51
Inputs
ObservationsMeasurementsDataProcess Outputs Measure-ment
ProcessInputs Outputs
2
tMeasuremen
2
Process
2
Total
Possible Sources of Variation
Long-term
ProcessVariation
Actual Process Variation Measurement Variation
Observed Process Variation
Short-term
ProcessVariation
Variation dueto instrument
Variationdue to
operator
Accuracy(Bias)
Precision(Measurement
Error)
-
7/29/2019 Attack the Variance.hurst-1
52/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 52
ProcessVariability
Does Sample A = B = C??
A B C
Technique #2Technique #1
sprocess= 5
stech #1= 3 stech#2= 1
What about the measurement system?
A B C A B C
What about the measurement system?
-
7/29/2019 Attack the Variance.hurst-1
53/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 53
How do I know where the variability is originating?
2
ilityReproducib
2
ityRepeatabil
2
tMeasuremen
2
tMeasuremen
2
Process
2
Observed
What about the measurement system?
Long-term
ProcessVariation
Actual Process Variation Measurement Variation
Observed Process Variation
Short-term
ProcessVariation
Variation dueto instrument
Variationdue to
operator
Accuracy(Bias)
Precision(Measurement
Error)
-
7/29/2019 Attack the Variance.hurst-1
54/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 54
GR&RMeasurement System
Interpretation
GR&R 10% Very capable
10% GR&R 20% Capable
20% GR&R 30% Marginally capable
GR&R 30% Not capable
%100)(15.5
% xT
sGRR
meas
%100)(
)(57.2% xSLx
sGRRmeas
Two-sided spec (T=USL-LSL) One-sided spec:
%GRR describes what % of your working range is taken up byvariability in your measurement system. The lower the %GRR, the
better you can distinguish actual process movement (often what yourcustomers are after) from measurement variability.
Meas.StdDev
GRR indicates method precision . . .
not accuracy (see note)
Gauge Repeatability & Reproducibility (%GRR)
-
7/29/2019 Attack the Variance.hurst-1
55/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 55
PolyStat, a polymer formulator, is analyzing a polymer sample for an
additive using NIR. They believe that they have optimized the methodand now want to perform a gauge R&R.
The two operators who will be running the test are asked to run thegauge samples.
Manufacturing has said that the upper spec is 0.8% and the lowerspec. is 0.3%. They have also indicated that the standard deviation ofthe process is 0.3%.
The Black Belt has picked samples she feels are representative of theprocess and has told the operators how the samples are to be run.
The results are in the Excel file GRR Example.xls.
Using Minitab, calculate the GRR based on the process capability andthe tolerance. Is this a good method?
If so, why? If not, why not and what can you do to fix it?
Breakout #3: GRR
-
7/29/2019 Attack the Variance.hurst-1
56/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 56
Open the file GRR Example.xls
Copy the .xls data into Minitab
Stack the data using the stack function in Minitab
Perform a GR&R using the Stat feature
Use the ANOVA approach, enter the process variance
(6 x 0.3 = 1.8)How good is the method?
Breakout #3: Instructor will demonstrate.
Breakout #4: Students will practice.
Instructions for Breakout Problem #3
-
7/29/2019 Attack the Variance.hurst-1
57/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 57
Have to stack columns before GRR
analysis in Minitab
Breakout #3: GRR Screen Shots
Operator 1 Operator 2 Sample Replicate
0.49 0.48 1 1
0.49 0.45 1 2
0.45 0.49 1 3
0.55 0.55 2 1
0.57 0.59 2 2
0.62 0.63 2 3
0.23 0.21 3 1
0.23 0.21 3 20.25 0.24 3 3
0.5 0.55 4 1
0.49 0.53 4 2
0.48 0.48 4 3
-
7/29/2019 Attack the Variance.hurst-1
58/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 58
1. Duplicate Samples
2. Set up GRR
analysis
Breakout #3: GRR Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
59/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 59
Enter Process variation
Whats this?
Well get to that later..
Breakout #3: GRR Screen Shots
-
7/29/2019 Attack the Variance.hurst-1
60/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 60
Part-to-PartReprodRepeatGage R&R
100
50
0
Percent
% Contribution
% Study Var% Process
0.10
0.05
0.00SampleRange
_R=0.0462
UCL=0.1191
LCL=0
Operator 1 Operator 2
0.6
0.4
0.2
SampleMean
__X=0.4483UCL=0.4957
LCL=0.4010
Operator 1 Operator 2
4321
0.6
0.4
0.2
Sample
Operator 2Operator 1
0.6
0.4
0.2
Operator
4321
0.6
0.4
0.2
Sample
Average
Operator 1
Operator 2
Operator
Gage name:
Date of study :
Reported by :
Tolerance:
Misc:
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Data by Sample
Data by Operator
Operator * Sample Interaction
Gage R&R (ANOVA) for Data
1. Good gaugerelative to
part variation
2. Repeatabilityis most ofgauge error
3. No interactionb/w operator
and part
4. Part 3 is low
Interpretation
Breakout #3: GRR Graphical Results
-
7/29/2019 Attack the Variance.hurst-1
61/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 61
Now enter process tolerance (USL-LSL)
Breakout #3: Screen Shots
Breakout #3: GRR Graphical Results
-
7/29/2019 Attack the Variance.hurst-1
62/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 62
Part-to-PartReprodRepeatGage R&R
160
80
0
Percent
% Contribution
% Study Var
% Process
% Tolerance
0.10
0.05
0.00Sam
pleRange
_R=0.0462
UCL=0.1191
LCL=0
Operator 1 Operator 2
0.6
0.4
0.2
SampleMean
__X=0.4483UCL=0.4957
LCL=0.4010
Operator 1 Operator 2
4321
0.6
0.4
0.2
Sample
Operator 2Operator 1
0.6
0.4
0.2
Operator
4321
0.6
0.4
0.2
Sample
Average
Operator 1
Operator 2
Operator
Gage name:
Date of study :
Reported by :
Tolerance:
Misc:
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Data by Sample
Data by Operator
Operator * Sample Interaction
Gage R&R (ANOVA) for Data
Measurement gauge R&Ris low relative to thetolerance (i.e. specs)
Repeatabilityerror takes upmore of thetolerance than
reproducibility.
The parts chosen werewell outside thetolerance and themethod was capable of
differentiating them.
Breakout #3: GRR Graphical Results
GRR Breakout #3: Results in Minitab Session Window
-
7/29/2019 Attack the Variance.hurst-1
63/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 63
What doesthis tell us?
Focus on the p-value:p
-
7/29/2019 Attack the Variance.hurst-1
64/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 64
Part-to-part variabilityaccounts for 158% of
the tolerance
Std. dev. parts/ Std. dev. gauge *1.41
Measurement gauge variabilityaccounts for 27.13% of the tolerance
GRR Breakout #3: Results in Minitab Session Window
-
7/29/2019 Attack the Variance.hurst-1
65/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 65
PolyStat is trialing some manufacturing changes. The preliminary dataindicate that they are now making better product, based on weatheringand tensile data.
The upper spec is 0.8% and the lower spec. is still 0.3%. They haveindicated that the standard deviation of the process is now 0.25%.
The same two operators are asked to run the GRR experiment with anew set of samples. Their results are in the Excel file GRR Example2.xls.
Using Minitab, calculate the GRR based on the process capability andthe tolerance. Is this a good method?
If so, why? If not, why not and what can you do to fix it?
Breakout #4: GRR
-
7/29/2019 Attack the Variance.hurst-1
66/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 66
Fishbone Diagram
FTIR & NIRIR Parameters
PeopleMaterialsMeasurements
Environment Methods Machines
Mole% X
Temperature
Film PressSample Placement
Standard
Quality
ContaminationSpectral Regions
Humidity
Sample Prep
Sample Loading
Multitasking
Spectral Quality
Statistical Model
Homogeneity
A great way to brainstorm potential factors that may affect the result
A cross-functional effort, allow enough time
Include all input, whether known or only suspected to affect the result
Better to have too many factors than too few . . . DOE will help screen
Example Fishbone diagramfor FTIR compositional test
method
-
7/29/2019 Attack the Variance.hurst-1
67/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 67
Pittcon March 2013
Section 3Identify sources of variation . . .
Then optimize!
SABIC Innovative Plastics
Analytical Technology
-
7/29/2019 Attack the Variance.hurst-1
68/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 68
Introduction to the Design of Experiment (DOE)
What is a DOE? When is it appropriate?
Visualizing the Experimental Space
- Factors and Levels, Coding
Importance of Replication and Randomization
Importance of Orthogonality and Blocking
Types of DOEs
Screening and Optimization
- Full and Fractional Factorial DOEs
Appendix
Second Order Designs
Additional Break-out Examples
Section 3: Outline
One at a Time Approach
-
7/29/2019 Attack the Variance.hurst-1
69/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 69
Box, Hunter, and Hunter. Statistics for Experimenters. John Wiley and Sons, Inc., 1978.
Challenge: Optimize yield of chemical reactionReaction time (t)
Reaction temperature (T)
One-at-a-time approach:maximum yield of 75 grams at 130 min and 225C
One-at-a-Time Approach
-
7/29/2019 Attack the Variance.hurst-1
70/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 70
Box, Hunter, and Hunter. Statistics for Experimenters. John Wiley and Sons, Inc., 1978.
- Time consuming- Expensive testing (time and supplies)
- No consideration of variable interactions
When variables are changedsimultaneously.
Global Maximum:
Yield of 91 grams65 min and 255C
Local Maximum:yield of 75 grams
130 min and 225C
One-at-a-Time Approach
-
7/29/2019 Attack the Variance.hurst-1
71/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 71
Structured process, formal plan Investigates the relationship between input and output
factors. One-at-a-time approach of controlling independent variables
and observing dependent variables
By contrast, what is a DOE?
Experimental methodology Multiple independent variables are controlled simultaneously
Structured experiments are conducted in a prescribed way Efficient and accurate analysis which determines the
significance and/or the mathematical relationships of factorsto a measured output.
What is Experimental Design?
-
7/29/2019 Attack the Variance.hurst-1
72/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 72
If preliminary work or expert knowledge reveals anobvious cause and a solution exists.
If: Root cause(s) cannot be found Further improvement is desired, after removing
root causes
Many potential factors affect the response You wish to quantify the relationship between the
factors and the response
Is a Designed Experiment Appropriate?
NO
YES
Visualizing the Experimental Space
-
7/29/2019 Attack the Variance.hurst-1
73/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 73
Challenge:How can we maximize
reaction yield?X + Y Z
Experimental variables from
team brainstorming:Factor A = TemperatureFactor B = TimeFactor C = pH
125C
50C
15min5min
Factor B
Factor A
4
125C
50C
15min
8
5minFactor B
Factor A
Factor C
Visualizing the Experimental Space
Visualizing the Experimental Space
-
7/29/2019 Attack the Variance.hurst-1
74/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 74
2 Factors, 2 Levels = SQUARE
3 Factors, 2 Levels = CUBE
Full Factorial:2^ 2 = 4 runs
Full Factorial:2^ 3 = 8 runs
# Levels^ #Factors = # Runs
Visualizing the Experimental Space
+1
-1+1-1
Factor B
Factor A
-1
+1
-1
+1
+1
-1Factor B
Factor A
Factor C
What kinds of Experimental Designs are there?
-
7/29/2019 Attack the Variance.hurst-1
75/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 75
Fractional FactorialFull Factorial
What information can we obtain using factorial designs?
Main effects of factors (Xi) and Effects of interactions (Xi * Xj)Variability associated with each factor and interactionEffects of experimental errorPredictive transfer functions, Example: y = b1(Xi) + b2(Xj) + b3(Xi*Xj)
2) Optimization DOE To validate screening DoE, determineoptimum input variables, characterize the input/output response surface(check for curvature) by adding levels or using different designs.
What kinds of Experimental Designs are there?
1) Screening DOE Identify statistically significant variables
I t ti E i Fi hb (C & Eff t)
-
7/29/2019 Attack the Variance.hurst-1
76/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 76
Interactive Exercise: Fishbone (Cause & Effect)Diagram
Means of organizing brainstorming ideas on the possible causes of
some stated outcome into major categories
Put down EVERYTHINGyoull filter the list later
Human Materials Machine
Method Nature Measurement
Outcome
R i t f DOE D i
-
7/29/2019 Attack the Variance.hurst-1
77/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 77
Several aspects of DOE design ensure the DOE analysis results
in statistically valid andpractically valid conclusions:
Replication
Randomization
Blocking
Balance & Orthogonality
Requirements for DOE Design
Replication and Experimental Error
-
7/29/2019 Attack the Variance.hurst-1
78/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 78
In order to determinestatistical significance of main effectsand interaction factors, the magnitude of these effects arecompared to the experimental error present in the system.
In order to estimate the error, experimental runs are replicated.
Where to add replicates? Where you want the mostinformation:
Factor A
Factor B +
+
+Factor C
Corner-pointreplication Provides error data across design
space Keeps experiment balanced
Factor A
Factor B +
+
+Factor C
Center-pointreplication More information about that areaof the design space Higher number of replicatesreduces confidence interval around
the estimate of error at that point
p p
Randomization is a MUST!
-
7/29/2019 Attack the Variance.hurst-1
79/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 79
123456789101112
-1+1-1+1-1+1
-1+1-1+1-1+1
-1-1+1+1-1-1
+1+1-1-1+1+1
-1-1-1-1+1+1
+1+1-1+1+1-1
StdOrder
FactorA
FactorB
FactorC
(Replicate Runs)
If the Run Order = Std Order,then any uncontrolled factor thatvaries with time (i.e. noise) willeffect response.
Example:
In an LC method, the temperature in thelaboratory can effect the chromatographicpeak resolution.
If the runs are not randomized, then thetemperature effect, which is not acontrolled factor, can effect the DOE.
Also, if all the replicates are run at the
end, then the experimental noise is notrepresentative of the true noise level.
The runs should be randomized. Replicates should be spread randomly
throughout the experiment.
RandomizedRun Order
83
121015
72946
11
Factor A
Factor B +
+
+
Factor C
Blocking
-
7/29/2019 Attack the Variance.hurst-1
80/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 80
What is Blocking?
Systematic way of dividing the runs into subsets. Maximum number
of runs that can be made under homogenous conditions.Why Block?
If there are factors which are believed to affect the response but areof no interest to the experimenter, these factors can confound theresults and produce erroneous conclusions.
By assigning a block to each level of this nuisance factor, the effect ofthis factor can be isolated from those of interest. (Time- shift, days,etc.)
When to Block?
When there is a significant risk of a nuisance factor confounding theexperimental results.. Without replication of these blocks, however,there is no ability to test for the significance of the blocking factor.
The cost of blocking is the ability to determine higher-order effects.Only block in screening DOEs- disregarding a source of variation willnot result in a robust design.
g
Full Factorial DOE: Balance and Orthogonality
-
7/29/2019 Attack the Variance.hurst-1
81/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 81
Run Order A B AB
1 (1) -1 -1 1
2 a 1 -1 -1
3 b -1 1 -1
4 ab 1 1 1
Balanced0 for each factor sum
This feature helps to simplify the analysis
X i =
Xi Balance and Orthogonality
Ensure uniform informationthroughout the design space
Ensure independent estimates
of factor effects
These conditions are satisfied
for full factorial designs
(Adapted from Mikel J. Harry. The Vision of Six Sigma. 1994. Page 18.9)
Unbalancedreplication
Not orthogonalEither missed levelor operational spacewas constrained.
g y
OrthogonalThis feature ensures the effects are independent
0 for all dot product pairs =X Xji
Full Factorial DOE: Cost vs. Benefit
-
7/29/2019 Attack the Variance.hurst-1
82/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 82
Numberof Factors
Numberof Runs
123456.
.
.10
248163264.
.
.1,024
Cost
$$$$$$
$$$$$$$$$
$$$$$$.
.
.$$$$$$$$$
Highest orderinteraction
123456.
.
.10
Number of 3-way interactions
0014916.
.
.119
Number of 2-wayinteractions
01361015.
.
.45
Question to consider:Do we have the time, money and resources
to run all these experiments,plusreplicates?
Usually-NO!!
2 ^ Factors = # Runs
Full Factorial DOE: Cost vs. Benefit
Fractional Factorial Designs for Screening DOEs
Screening DOE: Fractional Factorial
-
7/29/2019 Attack the Variance.hurst-1
83/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 83
123456
78
-1+1-1+1-1+1
-1+1
-1-1+1+1-1-1
+1+1
-1-1-1-1+1+1
+1+1
StdOrder
FactorA
FactorB
FactorC
Full Factorial 23 = 8 runs
Which runs do you choose to run? (or NOT?)
Remember what informationyou get from a full factorial
Grand mean: Y-bar
Main effects: A, B, C
2-Way interactions: AB, AC, BC
3-Way interaction: ABC
ABC
-1+1+1-1+1-1-1+1
Run only the runs for which ABC is -1
3-Way interaction terms are probably
NOT statistically significant
Half Fraction Factorial Design:2k-1 = 23-1 = 22 = 4 runs
Half Fraction design utilize half thenumber of runs of Full Factorial
Designs
Factor A
Factor B +
+
+
Factor C
-
7/29/2019 Attack the Variance.hurst-1
84/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 84
1) Identify number of factors
2) Identify levels of each factor
3) Calculate # runs for a full factorial# Full Factorial Runs = Levels ^ Factors
4) Select appropriate resolution
Which interactions can you tolerate?
5) Plan DOE using Minitab or other software
6) Finally... RUN EXPERIMENTS !!
Steps to designing a fractional factorial DOE:
-
7/29/2019 Attack the Variance.hurst-1
85/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 85
Resolution of Fractional Factorial Designs - Minitab
-
7/29/2019 Attack the Variance.hurst-1
86/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 86
Green = Res V+ Yellow = Res IV Red = Res III
Designs23-1
24-1
25-125-2
26-1
26-2
26-3
Runs4
8
168
3216
8
Alias StructureC = AB
D = ABC
E = ABCDD = AB, E = AC
F = ABCDEE = ABC, F = ACD
D = AB, E = AC, F = BC
ResolutionIII
IV
VIII
VIIV
III
Examples of fractional factorial designs
# Runs = Levels^ (Factors p)
Factorial then p = 1
Factorial then p = 2
Factorial then p = 3
-
7/29/2019 Attack the Variance.hurst-1
87/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 87
Fractional Factorial DOEConstruction
Breakout #5
Breakout #5: Ion Chromatography Example
-
7/29/2019 Attack the Variance.hurst-1
88/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 88
Water, extracted anions
Methylene chloride,dissolved polymer
Current Method: Volatile Chloride in PolymersHeat 5g sample to 300C
Collect vapor in H2
OAnalyze by Ion Chromatography
Proposed Method: Water Extraction of AnionsShake 2.5g sample in 20mL of CH2Cl2until dissolvedAdd 15mL H
2O &shake to extract
Allow emulsion time to settleAnalyze concentration in final
solution by IonChromatography
Heated Polymer Water,
volatile chloride
N2
g p y p
72% GRR!!
Breakout #5: Fractional Factorial DOE Construction
-
7/29/2019 Attack the Variance.hurst-1
89/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 89
Instructor will demonstrate how to set up DOE.
Then students will create the DOEClass will solve DOE together
1. Open Minitab
2. Create Factional Factorial DOE (2-Levels, 5-Factors)
with 1 center point, 1 block and 2 replicates at the corners
3. Label Xs: mass, volume, shake1, shake2, settle
4. Open file Breakout ppm Cl factorial doe.MPJ in Minitab
5. Analyze factorial design
Graph: regular residuals with a = 0.05
6. Identify any significant interactions (check p values)
7. Investigate ANOVA Main Effects and Interaction plots
Breakout #5: Fractional Factorial DOE Construction
Breakout #5: Create Factorial DOE (Screen Shots)
-
7/29/2019 Attack the Variance.hurst-1
90/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 90
Breakout #5: Create Factorial DOE (Screen Shots)
-
7/29/2019 Attack the Variance.hurst-1
91/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 91
Replicates let you estimate the error across designspace to help show whether factors are significant
Breakout #5: Enter Results (Screen Shots)
-
7/29/2019 Attack the Variance.hurst-1
92/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 92
Add factor labels
After running experiments,add the responses
Breakout #5: Analyze Factorial DOE (Screen Shots)
-
7/29/2019 Attack the Variance.hurst-1
93/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 93
Breakout #5: Analyze Factorial DOE (Screen Shots)
-
7/29/2019 Attack the Variance.hurst-1
94/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 94
TermsGraphs
This problem hasbeen worked out
before, and its knownnot to have any 3-wayinteractions. To savetime in class, choose
order of 2.
Analyze Factorial DOE - Residuals Plots
-
7/29/2019 Attack the Variance.hurst-1
95/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 95
302520151051
3
2
1
0
-1
-2
-3
Observation Order
StandardizedResid
ual
Versus Order(response is ppm Cl)
3210-1-2-3
99
95
90
80
70
60
50
40
30
20
10
5
1
Standardized Residual
Percent
Normal Probability Plot(response is ppm Cl)
0.90.80.70.60.50.4
3
2
1
0
-1
-2
-3
Fitted Value
StandardizedResid
ual
Versus Fits(response is ppm Cl)
2.41.20.0-1.2-2.4
9
8
7
6
5
4
3
2
1
0
Standardized Residual
Frequency
Histogram(response is ppm Cl)
No systematic variationduring DOE
Data are normallydistributed
Residuals are
indicative of noiseand therefore shouldbe normallydistributed andshould not exhibitsystematic variationwith time.
Patterns in theresiduals plots mayindicate a criticalfactor that is notbeing controlled!
Or that data
transformationmay be needed (i.e.Log(x))
Points falling on the line -
normally distributed
Look for trumpet
heteroscedastic data
Pareto Chart of Main Effects
-
7/29/2019 Attack the Variance.hurst-1
96/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 96
CE
BE
AC
E
BC
D
C
AD
BD
AB
DE
AE
CD
B
A
121086420
Term
Standardized Effect
2.11
A mass
B v olume
C shake1
D shake2
E settle
Factor Name
Pareto Chart of the Standardized Effects(response is ppm Cl, Alpha = 0.05)
The dotted line isthe boundarybetween significant(above the line) andinsignificant (belowthe line) accordingto alpha.
Quick Visual Confirmation of Significant Factors
Main Effects Plots
-
7/29/2019 Attack the Variance.hurst-1
97/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 97
Go to Stat >> DOE >> Factorial >> Factorial Plots . . .Click Setup to select plot type and terms.
1-1
0.75
0.70
0.65
0.60
0.55
1-1 1-1
1-1
0.75
0.70
0.65
0.60
0.55
1-1
mass
Mean
volume shake1
shake2 settle
Main Effects Plot for ppm ClData Means
Interaction Plots
-
7/29/2019 Attack the Variance.hurst-1
98/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 98
1-1 1-1 1-1 1-1
0.80
0.65
0.50
0.80
0.65
0.50
0.80
0.65
0.50
0.80
0.65
0.50
mass
volume
shake1
shake2
settle
-1
1
mass
-1
1
volume
-1
1
shake1
-11
shake2
Interaction Plot for ppm ClData Means
Go to Stat >> DOE >> Factorial >>Factorial Plots . . .
Click Setup to select plot typeand terms.
DOE Session Window Evaluation of Fit
-
7/29/2019 Attack the Variance.hurst-1
99/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 99
P < 0.05 indicates asignificant factor
Hmmmdoes this make
sense with what we knowfrom experience?
Shake1 * Settle is leastlikely to affect the ppm Cl
Evaluation of Fit
S H d d ill d
DOE Analysis Elimination of Insignificant Terms
-
7/29/2019 Attack the Variance.hurst-1
100/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 100
So . . . How do we drill downto the important factors?
Still
significantfactors
Least significant term . . .Eliminate it next!
p-values MAY change each time a
term is eliminated.
Af li i i i i ifi
Elimination of Insignificant Terms contd
-
7/29/2019 Attack the Variance.hurst-1
101/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 101
D
C
CD
B
A
121086420
Term
Standardized Effect
2.06
A mass
B v olume
C shake1
D shake2
Factor Name
Pareto Chart of the Standardized Effects
(response is ppm Cl, Alpha = 0.05)
. . . After eliminating insignificant terms
significantfactors
Good! Nosignificantlack of fit.
Transfer Function (in coded units):ppm Cl = 0.6530 (0.1022 x mass) + (0.0728 xvolume) (0.0197 x shake1 x shake2) + (0.0078x shake1) (0.0078 x shake2)
Why do weneed these?
Solve the Transfer Function in Uncoded Units
-
7/29/2019 Attack the Variance.hurst-1
102/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 102
Factor
Low
Setting
High
Setting
mass 2.5 7.5
volume 10 20
shake1 10 30
shake2 10 30
settle 5 15
If the actual low, high settings are . . .
Put into newworksheet
Solve the Transfer Function in Uncoded Units
-
7/29/2019 Attack the Variance.hurst-1
103/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 103
Factor levels now in actual units with the same run order as before
Copy responsesinto last column
Analyze Factorial Design andselect the significant factors
Solve the Transfer Function in Uncoded Units
-
7/29/2019 Attack the Variance.hurst-1
104/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 104
Transfer function in actual units:
PPM Cl = 0.5602 (0.04087 x mass) +(0.01456 x volume) (0.00020 xshake1*shake2) + (0.00472 x shake1) +(0.000316 x shake2)
Effect of Factors on Response Now Known . . .As long as the effect is linear for each factor
New details in Session window
D
C
CD
B
A
121086420
Term
Standardized Effect
2.05
A mass
B volume
C shake1
D shake2
Factor Name
Pareto Chart of the Standardized Effects(response is ppm Cl, Alpha = 0.05)
Screening DOE: Summary and Important Points toC id
-
7/29/2019 Attack the Variance.hurst-1
105/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 105
Helps you focus on the critical factors that influence the response
Points out important interactions between factors or directs your attention toother factors you hadnt considered
Transfer function should be verified experimentally by running selectedexperiments at the low and high settings to check predictive capability.
The screening model assumes that each factor has a linear effect on theresponse because only 2 factor levels were tested.
The screening model does not specifically address variance because eachcorner of the design space is tested only once.
You may need to perform a second screening DOE to include other factors orre-center the range.
The next sections will demonstrate how to strategically model the variance todevelop the most robust method.
An Optimization DOE on significant factors will help show any curvature inthe design space. . . . Next
Consider
Optimization DOE
-
7/29/2019 Attack the Variance.hurst-1
106/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 106
Why do an Optimization DOE? Add center-points for curvature
Re-center the design space based on screening results Focus only on significant factors Improve precision of the model by adding replicatesDetermine a transfer function between your xs and yourresponse
How is the experiment different? More than 2 levels to each factor Usually fewer factors Often additional replicates
Youre more knowledgeable now, and set a better design space
Does it take more time than a screening DOE?Usually doesnt take any more time, and sometimes takes less!
Optimization: Second Order Experimental Designs
-
7/29/2019 Attack the Variance.hurst-1
107/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 107
Corner Points-Assessment oflinear and 2-way
interactions
Center Points-Needed to determine ifcurvature is present. Replicating centerpoint is a common way to get at pure error.
Star (Axial) Points-For the assessment ofquadratic terms
Central Composite Designs Box-Behnken Designs
Exist only for
number of factors= 3 to 7
No Corner Points-Used in situationswhen design space isphysically constrainedand corner points arenot possible.
65
75
85
95
Rising
Ridge
-4.00-2.00
0.00
2.00
4.00
-4.00
-2.00
0.00
2.00
4.00
AB
Center points and star points areneeded to adequately model thebehavior of a response when thesystem includes curvature.
Example ofcurved
responsesurface
Breakout #6 RS Design with Optimization
-
7/29/2019 Attack the Variance.hurst-1
108/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 108
Method development specialists at Polystuff Products analyze the extent of
polymerization in lab-scale reactions by measuring the amount of residualmonomer following reaction in a stirred vessel. The scientists observe that, in
general, the rate of polymerization increases with increased stirrer speed.
However, when the stirrer speed gets too high, shear instability causes the
solids to coagulate out of suspension, reducing the polymerization rate. The
amount of suspended polymer can roughly be measured as turbidity by UV-Vis
absorption.
The scientists designed a central composite design DOE with two factors
(stirrer speed and mass of initial monomer) measuring two results (residual
monomer and turbidity) find the optimal settings for their process. The goal is
to achieve a residual monomer level between 410 ppm and 427 ppm, and
maximum relative turbidity between 0.7 and 0.88.
Experimental data can be found in the breakout file named:
Breakout-6 CCD Optimization-DOE.MPJ
Breakout #6 RS Design with Optimization
Optimization: Create RS (Response Surface) Design
-
7/29/2019 Attack the Variance.hurst-1
109/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 109
Optimization: Create RS (Response Surface) Design
-
7/29/2019 Attack the Variance.hurst-1
110/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 110
After runningexperiments,add the responses
Factorsettings
Optimization: Create RS (Response Surface) Design
Optimization: Analyze RS Design
-
7/29/2019 Attack the Variance.hurst-1
111/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 111
Include all terms(main effects,
interactions,quadratic)
Residuals plots can begenerated as shown
before
Optimization: Analyze RS Design
Analyze oneresponse at a time
Optimization: Analyze RS Design
-
7/29/2019 Attack the Variance.hurst-1
112/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 112
Now eliminate least significant terms one-by-one,leaving in the main effects
Mass * Mass is least significant
Lack of Fit is not significant.
Good
Stirrer * Mass is significant
Response = Residual Monomer
The regression is significant.
Optimization: Analyze RS Design
-
7/29/2019 Attack the Variance.hurst-1
113/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 113
Residual Monomer = 406.774 + (9.780 x stirrer) +(46.279 x Mass) (9.439 x stirrer x mass)
Final Transfer Function
Stirrer x Mass remains
significant. Main effects
must be kept.
Response = Residual Monomer
Optimization: Analyze RS Design
-
7/29/2019 Attack the Variance.hurst-1
114/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 114
Response = Turbidity
Turbidity = -0.01829 + (0.12011 x stirrer) +(0.18035 x Mass) (0.00649 x stirrer2)
Final Transfer Function
Stir, Mass, and Stir2
are all significant, but what
about the constant?
After drilling down to thecritical factors . . .
Optimization: Analyze RS Design
Optimization: Plot the Response Surface
-
7/29/2019 Attack the Variance.hurst-1
115/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 115
RS Plots help visualize the interacting
effect of each factor on the response
Contour Plot
flat graph with coloredtopographical lines Wireframe Plot3-D projection of curve
Each plot has Setup
options (examples below)
Specify which
response andfactors to plot
Specify how to
handle otherfactors if more
than 2 factors
exist.
C t Pl t f R id l M Sti d M Wi f Pl t f R id l M Sti d M
Optimization: Plot the Response Surface
-
7/29/2019 Attack the Variance.hurst-1
116/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 116
. . . But SO WHAT?
I want to know how to set all the knobs on my method tooptimize both responses simultaneously!
2.152.051.951.851.752.5
0.51.65
3.5Mass
0.6
4.5
0.7
1.555.5
0.8
0.9
6.51.45
7.5 8.5
Turbidity
1.359.510.5
1.2511.5Stirrer
Wireframe Plot of Turbidity vs. Stirrer and Mass
0.5292680.5648590.6004510.6360420.6716330.7072240.7428150.7784070.8139980.8495890.8851800.920772
3 4 5 6 7 8 9 10 11
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0
2.1
Stirrer
Mass
Contour Plot of Turbidity vs. Stirrer and Mass
396.719403.990411.260418.531425.802433.072
440.343447.614454.885462.155469.426476.697
3 4 5 6 7 8 9 10 11
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0
2.1
Stirrer
Mas
s
Contour Plot of Residual Monomer vs. Stirrer and Mass
2.152.051.951.851.752.5
3901.65
3.5
400
Mass
410
420
4.5
430
440
1.555.5
450
460
470
6.5
480
1.457.5 8.5
1.35
Residual Monomer
9.510.51.25
11.5Stirrer
Wireframe Plot of Residual Monomer vs. Stirrer and Mass
ResidualMonomer
Turbidity
Minitab 15 has snazzier plots than Minitab 12
Optimization: Optimize all Responses
-
7/29/2019 Attack the Variance.hurst-1
117/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 117
Optimization: Optimize all Responses
-
7/29/2019 Attack the Variance.hurst-1
118/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 118
CurHigh
Low1.0000D
Optimal
d = 1.0000
Targ: 427.0
Residual
y = 427.0000
d = 1.0000
Maximum
Turbidit
y = 0.9031
1.0000Desirability
Composite
1.2757
2.1243
2.7574
11.2426MassStirrer
[7.6026] [2.1243]
Optimal FactorSettings
Predicted
Responses
Drag the vertical red lines
left and right to changethe factors and see theeffect on each response
Now you have it all!
To reset:Right click,Reset tooptimalsettings
Course 1 Summary
-
7/29/2019 Attack the Variance.hurst-1
119/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 119
Course 1 Summary
There are several tools to help understand your data
(hypothesis tests, ANOVA, GRR, etc.) These tools can indicate sources of variation or
equivalence of data sets
Several DOE strategies are available to identify the
factors that affect the response Screening DOE simplest design for quickly probing
linear effects for multiple factors
Optimization DOE more complex design forunderstanding higher order interactions.
Proper use of these concepts can help you trulyoptimize your methods.
Revisiting Students Course Expectations
-
7/29/2019 Attack the Variance.hurst-1
120/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 120
Revisiting Students Course Expectations
Did you get what you came for?
Additional references
1. Basic Statistics, Tools for Continuous Improvement. Mark J. Kiemele, Stephen
-
7/29/2019 Attack the Variance.hurst-1
121/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 121
1. Basic Statistics, Tools for Continuous Improvement. Mark J. Kiemele, StephenR. Schmidt, Ronald J. Berdine. Air Academy Press, 1999. ISBN 1-880156-06-7
2. Chemometrics: Data Analysis for the Laboratory & Chemical Plant. Richard G.Brereton. Wiley, 2003. ISBN 0-471-48978-6
3. Experimental Design: A chemometric approach. Stanley N. Deming, StephenL. Morgan. Elsevier, 1987. ISBN 0-444-42734-1
4. Statistics: A Guide to the Use of Statistical Methods in the Physical Sciences.J.R. Barlow. Wiley, 1989. ISBN 0-471-92295-1
5. Statistics and Chemometrics for Analytical Chemistry. Jane Miller. PrenticeHall, 2005. ISBN 0-131-29192-0
6. Statistics for Analytical Chemistry. Jane C. Miller, James N. Miller. PrenticeHall, 1993. ISBN 0-130-30990-7
7. Statistics for Experimenters, An Introduction to Design, Data Analysis, andModel Building. George E. P. Box, William G. Hunter, and J. Stuart Hunter.Wiley, 1978. ISBN 0-471-09315-7
8. Understanding Industrial Designed Experiments. Stephen R. Schmidt, Robert
G. Launsby. Air Academy Press, 1994. ISBN 978-18801560329. http://science.widener.edu/svb/stats/stats.html
10.www.statease.com
11.www.minitab.com/resources
12.www.itl.nist.gov
Acknowledgements
-
7/29/2019 Attack the Variance.hurst-1
122/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 122
Acknowledgements
Pittcon Short Course Committee
SABIC Innovative Plastics Analytical Technology
Statistics Group at GE Global Research Center (Angie Neff,
Martha Gardner)
Appendix
-
7/29/2019 Attack the Variance.hurst-1
123/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 123
Pittcon March 2013
Course 1 Appendix
SABIC Innovative Plastics
Analytical Technology
ppe d
Anther DOE Design: Mixture (available with Design Expert software)Appendix
-
7/29/2019 Attack the Variance.hurst-1
124/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 124
Simply another class of DoEsRequirements:
Components dependent oneach other
Response is based on ratioof components
If not, investigate response
surface designs
Types of mixture DoEs: Simplex Identical factor ranges for
each component Non-constrained design D-Optimal Highly constrained design
Breakout: full factorial DOE practice
Appendix
-
7/29/2019 Attack the Variance.hurst-1
125/126
Slide-Attack the Variance, Course 1 (Pittcon 2013) 125
Example: Extraction of Pb from a soil matrix.
The primary factors considered are Time, Temp, and Acid Conc.
Create a full factorial DOE:23 = 8 runs
with replicates at each point = 16 total runs
Create the DOE in Minitab
Input the experimental results from the file:DOE ppm Pb.xls
Label columns and analyze the data.
p
(coded and uncoded data are available on
separate worksheets within the Excel file)
STD
FACTORS
Consider a Full 24 factorial design (16 Runs)
Challenge: If you can only afford to run 8
Breakout: fractional DOE practice Appendix
-
7/29/2019 Attack the Variance.hurst-1
126/126
STD
Order A B C D ABCD
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
1. Create DOE by hand, filling in 1 & 1 levels
2. Compute the dot product for ABCD3. Choose runs for which ABCD = 1 or1
STD
Order A B C D=ABC ABCD
1
2
3
45
6
7
8
FACTORS
This is the same as using the Design
GeneratorD = ABC:
1.Construct a 23 design with A, B, and C
2 Set D=ABC and compute dot product
C a e ge If you can only afford to run 8runs, which 8 do you choose?
For a 24 full factorial:
Grand mean: X-bar
Main effects: A, B, C, D
2-Way interactions: AB, AC, AD, BC, BD,CD
3-Way interactions: ABC, ABD, ACD, BCD
4-Way interaction: ABCD
16 items estimated from the 16 runs