Six Sigma Champion Training 1
Measurement Systems Analysis
• Champion Training
Improve
Analyze
Measure
Control
Six Sigma Champion Training 2
Learning Objectives
• Understand the language of Measurement
• Show the importance of Measurement
• Walk away knowing how to perform a Gage R&R and how to interpret results
• Share some lessons learned
Six Sigma Champion Training 3
• They are subject to Variation.
• What could be the source of this variation?
• Why do Measurements Vary?
CLASS EXERCISE : Break into teams. Do a cause-effect Diagram to determine the causes of variation in Measurement.
Since Measurement systems represent a sub-process within a process...
Six Sigma Champion Training 4
Sources of Measurement Variation
Measurement System Error
Temp Fluxctuation
Line Voltage Variation
Vibration
Cleanliness
Humidity
Algorithm Instability
Electrical Instability
Wear
Mechanical Integrety
Operator Technique
Standard Procedure
Suff icient Work time
Maintenance Standard
Calibration Frequency
Operator Training
Ease of use
Density
Conductivity
Hardness
Corrosion
Weight
Dimension
Temperature
Cleanliness
Wear
Stability
Resolution
Calibration
Precision
Design
Temperature
Cleanliness
VisionDexterity
Know ledgeCoordination
SpeedInterpretation
Calibration ErrorAttention
FatigueProcedure
Men
Machines
Materials
Methods
Measurements
Environment
Measurement System C&E Matrix
Six Sigma Champion Training 5
Possible Sources of Process Variation
Long-term
Process Variation
Short-term
Process Variation
Variation
w/i sample
Actual Process Variation
Stability LinearityRepeatability Accuracy
Variation due
to gage
Variation due
to operators
Measurement Variation
Observed Process Variation
We will look at “repeatability” and “reproducibility” as these are the primary contributors to measurement error.We will look at “repeatability” and “reproducibility” as these are the primary contributors to measurement error.
Reproducibility
Six Sigma Champion Training 6
Knowledge to be obtained
• How big is the measurement error?
• What are the sources of measurement error?
• Is the gage stable over time?
• Is the gage capable for this process?
• How do we improve the measurement system?
Six Sigma Champion Training 7
Sources of Variation
Product Variability
(Actual variability)
Product Variability
(Actual variability)
MeasurementVariability
MeasurementVariability
Total Variability(Observed variability)
Total Variability(Observed variability)
Six Sigma Champion Training 8
Effects of Measurement Error
AveragesAverages
VariabilityVariability
total product measurement
total product measurement2 2 2
Measurement System Bias —
Determined through “Accuracy Study”
Measurement System Variability —
Determined through “R&R Study”
Six Sigma Champion Training 9
If you want to decrease your gage error take advantage of the standard error square root of the sample:
Example: Gage error of 50% can be cut in half if your point estimate is a sample of 4 data points.
Example: Gage error of 50% can be cut in half if your point estimate is a sample of 4 data points.
THIS IS USED AS A SHORT TERM APPROACH TO PERFORM A STUDY, BUT YOU MUST FIX THE GAGE.
n= sample size
xx
n
Work Around Gage Error
Six Sigma Champion Training 10
Terminology
• Location related terms:– True value– Bias– Linearity
• Stability (over time)• Variation related terms
– Repeatability– Reproducibility– Linearity
Six Sigma Champion Training 11
• True value:– Theoretically correct value – unknown and unknowable– Reference standards– NIST standards
• Bias– Distance between average value of all measurements
and true value– Amount gage is consistently off target– Systematic error or offset
Six Sigma Champion Training 12
BIAS — Is the difference between the observed average of the measurement and the reference value. The reference-value is the value that serves as an agreed-upon reference. The reference value can be determined by averaging several measurements with a higher level (e.g., metrology lab) of measuring equipment.
Warning: Don’t assume your metrology reference is gospel. Observed
Average Value
ReferenceValue
ACCURACY IS THE SAME AS BIAS
BIAS Definition
Six Sigma Champion Training 13
LinearityDifference in the accuracy values of a gage through the expected operating range of the gage
5040302010
55
45
35
25
15
5
Standard
Tria
ls
R-Squared = 0.981
Y = 0.934227 + 0.994959X
Regression Plot
5040302010
55
45
35
25
15
5
Standard
Tria
ls
R-Squared = 0.982
Y = 0.245295 + 0.99505X
Linearity is Not Good
Good Linearity Bad Linearity
Six Sigma Champion Training 14
Stability
• The distribution of measurements remains constant and predictable over time for both mean and standard deviation
• Total variation in the measurements obtained with a gage, on the same master or master parts, when measuring a single characteristic over an extended time period.
• Evaluated using a trend chart or multiple measurement analysis studies over time
Six Sigma Champion Training 15
Stability (drift) Definition
Stability — Is the total variation in the measurement obtained with a measurement system (test / gage ) on the same master parts when measuring a single characteristic over an extended time period.
Time-1
Time-2
time
Magnitude
StabilityPoints to the frequency of Mean center CalibrationPoints to the frequency of Mean center Calibration
Six Sigma Champion Training 16
MS G O2 2 2
• Total variation in the measurement system• Measure of natural variation of repeated
measurements• Terms: Random Error, Spread, Test/Retest
error• Repeatability and Reproducibility
Six Sigma Champion Training 17
G
G
R
d
2*
Repeatability
• The inherent variability of the measurement system• Variation in measurements obtained with a gage when
used several times by one operator while measuring a characteristic on one part.
• Estimated by the pooled standard deviation of the distribution of repeated measurements
• Repeatability is less than the total variation of the measurement system
Six Sigma Champion Training 18
Repeatability Definition
REPEATABILITY
Repeatability — The variation in measurementsobtained with one measurement instrument when used several times by one appraiser whilemeasuring the identical characteristic onsame part.
Six Sigma Champion Training 19
O
O
R
d
2*
Reproducibility
• Operator variability of the measurement system• Variation in the average of the measurements
made by different operators using the same gage when measuring a characteristic on one part
• Must be adjusted for gage variation• Reproducibility is less than the total variation of
the measurement system
Six Sigma Champion Training 20
Reproducibility — Is the variation in the average of the measurements made by different appraisers using the same measuring instrument when measuring the identical characteristic on the same part.
Reproducibility
Operator-A
Operator-C
Operator-B
Reproducibility Definition
Six Sigma Champion Training 21
The Nature of Process Variation
54321
4321
Precise but not AccuratePrecise but not Accurate
Accurate but not PreciseAccurate but not Precise
. . . . . .Test equipment MUST be a least 10 times more accurate & precise then what’s being tested
Rule of thumb:
Six Sigma Champion Training 22
Measurement System DiscriminationLeast count should be at most one-tenth of the total
process capability or tolerance (6 sigma)
– Process capability 10 Max Least count 1Part to Part variation must be greater than the smallest
unit of measureRange control chart provides best indication of
inadequate discrimination
– Occurs when only 1,2, or 3 possible values for the range within the control limits exists
Number of Distinct Categories equals part sigma/ total
gage sigma 1.41.
Least count should be at most one-tenth of the total process capability or tolerance (6 sigma)
– Process capability 10 Max Least count 1Part to Part variation must be greater than the smallest
unit of measureRange control chart provides best indication of
inadequate discrimination
– Occurs when only 1,2, or 3 possible values for the range within the control limits exists
Number of Distinct Categories equals part sigma/ total
gage sigma 1.41.
Six Sigma Champion Training 23
Measurement System Capability
Gage Capability
Addresses what percent of the tolerance or process capability is taken up by measurement error.
Best case: 10% Acceptable: 30% Includes both repeatability and reproducibility
– Operator Unit Trial Experiment
GR RTolerance
MS&. *
515 Usually expressed
as percent
Usually expressed as percent
Six Sigma Champion Training 24
CUSL LSL
pAct
Act
6 Act Obs MS 2 2
CUSL LSL
pAct
Obs MS
6 2 2
where
CpMeasurement Error
Effect on Capability Index
• We know that
• Therefore:
Six Sigma Champion Training 25
R&R Effect on Capability
0.0
1.0
2.0
3.0
4.0
5.0
6.0
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0
Observed Cp
Ac
tua
l Cp 0%
10%
20%
30%
40%50%
60%
70%
%R&R
70% 60% 50%
40%
30%
10%
Six Sigma Champion Training 26
Types of R&R Studies
• Variable Gage R&R– Numbers– Units of measure
• Attribute Gage R&R– Subjective (cosmetic defects)– Scatter of defects– feel/visual
Six Sigma Champion Training 27
The Inspection Exercise
The Necessity of Training Farm Hands for First
Class Farms in the Fatherly Handling of Farm Live
Stock is Foremost in the Eyes of Farm Owners.
Since the Forefathers of the Farm Owners Trained
the Farm Hands for First Class Farms in the
Fatherly Handling of Farm Live Stock, the Farm
Owners Feel they should carry on with the Family
Tradition of Training Farm Hands of First Class
Farmers in the Fatherly Handling of Farm Live
Stock Because they Believe it is the Basis of Good
Fundamental Farm Management.
Task: Count the number of times the 6th letter of the alphabet appears in the following text.
Six Sigma Champion Training 28
Try To Always Convert Attribute To VariablesTry To Always Convert Attribute To Variables
Examples:
End Disk Height
Likert Scale
Leak Rate (go/no go)
Mass Spec
Convert Data
Six Sigma Champion Training 29
Basic Terms
EV= Equipment Variation (Repeatability)
AV= Appraiser Variation (Reproducibility)
R&R= Repeatability & Reproducibility
PV= Part Variation
TV= Total Variation of R&R and PV
K1-Trial, K2-Operator, & K3-Part Constants
EV= Equipment Variation (Repeatability)
AV= Appraiser Variation (Reproducibility)
R&R= Repeatability & Reproducibility
PV= Part Variation
TV= Total Variation of R&R and PV
K1-Trial, K2-Operator, & K3-Part Constants
Six Sigma Champion Training 30
Generally two or three operators
Generally 10 units to measure
Each unit is measured 2-3 times by each operator
Gage R&R study
Six Sigma Champion Training 31
Preparation for a Measurement Study
• Determine if reproducibility is an issue. If it is, select the number of operators to participate.
• Operators selected should normally use the measurement system.
• Select samples that represent the entire operating range.• Gage must have graduations that allow at least one-tenth of
the expected process variation.• Insure defined gaging procedures are followed.• Measurements should be made in random order.• Study must be observed by someone who recognizes the
importance of conducting a reliable study.
Six Sigma Champion Training 32
Procedure for Performing R&R Study
• Calibrate the gage, or assure that it has been calibrated.• Have the first operator measure all the samples once in random
order.• Have the second operator measure all the samples once in
random order.• Continue until all operators have measured the samples once
(this is Trial 1).• Repeat above steps for the required number of trials.• Use GR&R form to determine the statistics of the study.
– Repeatability, Reproducibility & %GR&R– Standard deviations of each of the above– % Tolerance analysis
• Analyze results and determine action, if any.
Six Sigma Champion Training 33
Guidelines
% R&R Results<5% No issues
10% Gage is OK
10% – 30% Maybe acceptable based upon importanceof application, and cost factor
Over 30% Gage system needs improvement/correctiveaction
% R&R Results<5% No issues
10% Gage is OK
10% – 30% Maybe acceptable based upon importanceof application, and cost factor
Over 30% Gage system needs improvement/correctiveaction
Variable Gage R&R
Six Sigma Champion Training 34
PT1 PT2 PT3 PT4 PT5 PT6 PT7 PT8 PT9 PT10OP/TRIAL
0.65 1.00 0.85 0.85 0.55 1.00 0.95 0.85 1.00 0.60 A1
0.60 1.00 0.80 0.95 0.45 1.00 0.95 0.80 1.00 0.70 A2
0.55 1.05 0.80 0.80 0.40 1.00 0.95 0.75 1.00 0.55 B1
0.55 0.95 0.75 0.75 0.40 1.05 0.90 0.70 0.95 0.50 B2
0.50 1.05 0.80 0.80 0.45 1.00 0.95 0.80 1.05 0.85 C1
0.55 1.00 0.80 0.80 0.50 1.05 0.95 0.80 1.05 0.80 C2
Xbar & R Minitab Example
Using Aiag49:mtw Data File
Specification: 0.6 - 1.0 mm
Process Variation: 1.6 mm
Reference QS Measurement System Analysis Manual
Gasket Thickness Study
Six Sigma Champion Training 35
Misc:
Tolerance:Reported by :
Date of study :Gage name:
1.11.00.90.80.70.60.50.40.3
321
Xbar Chart by Operator
Sam
ple
Mea
n
X=0.80753.0SL=0.8796
-3.0SL=0.7354
0.15
0.10
0.05
0.00
321
R Chart by Operator
Sam
ple
Ran
ge
R=0.03833
3.0SL=0.1252
-3.0SL=0.000
10 9 8 7 6 5 4 3 2 1
1.11.00.90.80.7
0.60.50.4
Gasket
OperatorOperator*Gasket Interaction
Ave
rage
123
321
1.11.00.90.80.7
0.60.50.4
Operator
Response by Operator
10 9 8 7 6 5 4 3 2 1
1.11.00.90.80.70.60.50.4
Gasket
Response by Gasket
%Total Var%Study Var%Process %Toler
Part-to-PartReprodRepeatGage R&R
200
100
0
Components of Variation
Perc
ent
Gage R&R (Xbar/R) for Thickness
Six Sigma Champion Training 36
Gage R&R Study for Thickness – XBar/R Method
Source Variance StdDev 5.15*Sigma
Total Gage R&R 2.08E-03 0.045650 0.235099 Repeatability 1.15E-03 0.033983 0.175015 Reproducibility 9.29E-04 0.030481 0.156975 Part-to-Part 3.08E-02 0.175577 0.904219 Total Variation 3.29E-02 0.181414 0.934282
Source %Contribution %Study Var %Tol %Process
Total Gage R&R 6.332 25.164 58.77 14.69 Repeatability 3.509 18.733 43.75 10.94 Reproducibility 2.823 16.802 39.24 9.81 Part-to-Part 93.668 96.782 226.05 56.51 Total Variation 100.000 100.000 233.57 58.39
Number of distinct categories = 5
Six Sigma Champion Training 37
Calculation Explanation
5.15 Sigma = 5.15 the factor standard deviation. 5.15 was developed empirically to approximate the gage population distribution variation.
% Contribution = Percent contribution of each factor based upon the variance. Repeatability = 100 repeatability variance/ total variation variance.
% Study Variation = 5.15 the factor standard deviation divided by 5.15 the total variation standard deviation. Repeatability = 100 5.15 repeatability standard deviation/ 5.15 total variation standard deviation.
% Tolerance = 5.15 the factor standard deviation divided by the tolerance. Repeatability = 100 5.15 repeatability standard deviation/tolerance.
5.15 Sigma = 5.15 the factor standard deviation. 5.15 was developed empirically to approximate the gage population distribution variation.
% Contribution = Percent contribution of each factor based upon the variance. Repeatability = 100 repeatability variance/ total variation variance.
% Study Variation = 5.15 the factor standard deviation divided by 5.15 the total variation standard deviation. Repeatability = 100 5.15 repeatability standard deviation/ 5.15 total variation standard deviation.
% Tolerance = 5.15 the factor standard deviation divided by the tolerance. Repeatability = 100 5.15 repeatability standard deviation/tolerance.
Six Sigma Champion Training 38
Calculation Explanation
% Process Variation = 5.15 x the factor standard deviation divided by the process variation. Repeatability = 100 x 5.15 repeatability standard deviation/ process variation.
Number of Distinct Categories = part standard deviation divided by the total gage R&R standard deviation times 1.41.
% Process Variation = 5.15 x the factor standard deviation divided by the process variation. Repeatability = 100 x 5.15 repeatability standard deviation/ process variation.
Number of Distinct Categories = part standard deviation divided by the total gage R&R standard deviation times 1.41.
Six Sigma Champion Training 39
Attribute Data ExampleUsing A_grr.xls file
• Metric is the % error against known population deemed good by local experts
• Attribute legend can be the defect codes• If appraiser % is less than 100% training is required, focus
on area of weakness• 100% is the target for screen effectiveness• Use this to prove measurement system capability prior to
task assignment• Select the 5 vital few (80-20 rule) to conduct GR&R
Six Sigma Champion Training 40
Instructions:
1) The following spreadsheet is used to calculate an Attribute GR&R Effectiveness, in which up to 30 samples can be evaluated, using 2 or 3 operators.
2) In the worksheet fill in the appropriate information in the Scoring Report section andenter the type of Attributes you are evaluating in the Attribute Legend section. YOU MUST ENTER THE INFORMATION IN THE ATTRIBUTE LEGEND SECTION OR THE SPREADSHEET WILL NOT WORK. The attributes can be either alpha or numeric, e.g. Yes, No; pass, fail; go, stop; or 1, 2. You must be consistent throughout the form and spell properly, anything will work as the spreadsheet compares what is in each cell.
3) If you or an expert has selected samples to be evaluated and you know what attributes thesesamples are, enter this information in the Attribute sample column. This will enable you to determine how well each operator can evaluate a set of samples against a known standard. You do notneed to enter information in this column for the spreadsheet to work.
4) You do not have to specify how many operators or the # of samples that you will be evaluating during the test. Simply enter the data into the spreadsheet under the specific operator. Rememberthe attributes must be spelled properly or the spreadsheet will not analyze the data correctly.
5) To print a copy of the report click on the Print Report icon.6) To delete the data in the spreadsheet, click on the Delete Data icon.7) To delete all and begin a new test, click on the Delete All icon8) To see a Demo of the Attribute GR&R Effectiveness spreadsheet, click on the Demo icon.
Move around the spread sheet to see the data. When you are finished click the Delete All icon to delete all data to begin entering your own data.
Attribute Gage R & R Effectiveness
Six Sigma Champion Training 41
Attribute Gage R & R Effectiveness
SCORING REPORTAttribute Legend DATE:
1 pass NAME: 3/10/96
2 fail PRODUCT: Allied Employee
SBU: 3313 Spark Plug
TEST CONDITIONS: F&SP
Known Population Operator #1 Operator #2 Operator #3 Y/N Y/NSample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2 Agree Agree
123456789
101112131415161718192021222324252627282930
% APPRAISER SCORE(1) -> #DIV/0! #DIV/0! 0%
% SCORE VS. ATTRIBUTE(2) -> Known Known Known
SCREEN % EFFECTIVE SCORE(3) -> #DIV/0!
SCREEN % EFFECTIVE SCORE vs. ATTRIBUTE (4) -> #DIV/0!
Six Sigma Champion Training 42
Attribute Gage R & R Effectiveness
SCORING REPORTAttribute Legend DATE: 3/10/96
1 pass NAME: Allied Employee
2 fail PRODUCT: 3313 Spark Plug
SBU: 3313 Spark Plug
TEST CONDITIONS: F&SP
Known Population Operator #1 Operator #2 Operator #3 Y/N Y/NSample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2 Agree Agree
1 pass pass pass pass pass fail fail N N2 pass pass pass pass pass fail fail N N3 fail fail fail fail pass fail fail N N4 fail fail fail fail fail fail fail Y Y5 fail fail fail pass fail fail fail N N6 pass pass pass pass pass pass pass Y Y7 pass fail fail fail fail fail fail Y N8 pass pass pass pass pass pass pass Y Y9 fail pass pass pass pass pass pass Y N
10 fail pass pass fail fail fail fail N N11 pass pass pass pass pass pass pass Y Y12 pass pass pass pass pass pass pass Y Y13 fail fail fail fail fail fail fail Y Y14 fail fail fail pass fail fail fail N N15 - - - - - - -
16 - - - - - - -
17 - - - - - - -
18 - - - - - - -
19 - - - - - - -
20 - - - - - - -
21 - - - - - - -
22 - - - - - - -
23 - - - - - - -
24 - - - - - - -
25 - - - - - - -
26 - - - - - - -
27 - - - - - - -
28 - - - - - - -
29 - - - - - - -
30 - - - - - - -
% APPRAISER SCORE(1) -> 100.00% 78.57% 100.00%
% SCORE VS. ATTRIBUTE(2) -> 78.57% 64.29% 71.43%
SCREEN % EFFECTIVE SCORE(3) -> 57.14%
SCREEN % EFFECTIVE SCORE vs. ATTRIBUTE(4) -> 42.86%
Six Sigma Champion Training 43
CalculationKnown Population Operator #1 Operator #2 Operator #3 Y/N Y/NSample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2 Agree Agree
1 1 1 1 1 1 2 2 FALSE FALSE2 1 1 1 1 1 2 2 FALSE FALSE3 2 2 2 2 1 2 2 FALSE FALSE4 2 2 2 2 2 2 2 TRUE TRUE5 2 2 2 1 2 2 2 FALSE FALSE6 1 1 1 1 1 1 1 TRUE TRUE7 1 2 2 2 2 2 2 TRUE FALSE8 1 1 1 1 1 1 1 TRUE TRUE9 2 1 1 1 1 1 1 TRUE FALSE10 2 1 1 2 2 2 2 FALSE FALSE11 1 2 1 1 1 1 1 FALSE FALSE12 1 2 1 1 1 1 1 FALSE FALSE13 2 2 2 2 2 2 2 TRUE TRUE14 2 2 2 1 2 2 2 FALSE FALSE15161718192021222324252627282930
% Appraiser Score 85.71% 78.57% 100.00% 42.9% 28.6%Agreements 12 11 14 6 4
Total Sample 14 14 14 14 14
Six Sigma Champion Training 44
Attribute Gage R&R Workshop
• Perform attribute Gage R&R study using M&M's.– Determine the defects by looking for consistent color in
the M&M's, clear markings (M's), and roundness.– Use 3 operators/inspectors
• Complete attribute GR&R analysis and report results (30 minutes).
• Improve inspection criteria, rerun attribute Gage R&R study/analysis and report results (30 minutes).
Six Sigma Champion Training 45
ANY QUESTIONS OR COMMENTS
Do You Understand?
• The language of Measurement ?
• The importance of Measurement?
• How to perform a Gage R&R Study and how to interpret results ?
• Use Minitab to analyze GR&R results?
Top Related