1.5 LSS Quality Files Bus. Risk Management (1)
-
Upload
murrell-j-rizon -
Category
Documents
-
view
219 -
download
0
Transcript of 1.5 LSS Quality Files Bus. Risk Management (1)
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
1/81
ASU Lean Six Sigma Green Belt DMAIC
Measure Phase
Introduction to Measure
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
2/81
Is the team operating within the context of the project charter?project scopegoal statement
project plan
Introduction to Measure
Staying on track ------------------------------------ Maintaining Perspective
1.0Define
Opportunities
2.0Measure
Performance
3.0Analyze
Opportunity
4.0Improve
Performance
5.0Control
PerformanceDefine Oppor tunit ies
Measure Perform ance
Analyze Oppor tuni ty
Improve
Performance
Contro l
Performance
Has fact- based decision making been consistently applied? Are the teams assumptionscontinuously validated?Is business risk being actively managed?Is leadership informed & on board with the teams findings & conclusions? Can the likelihood of success be improved by revisiting previous conclusions or analysis?
ASU Lean Six Sigma Green Belt DMAIC
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
3/81
Changing Our Perspective
Select the Y that the customer uses to judge your performance
Start with the customer
Measure the same as the customer does
Understand the variation in the output (what your processproduces that is of value to the customer)
Use data to find the process keys that drive the variation
Outside-In Focus Drives DMAIC Success
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
4/81
What Is A Measure?
Measurement can be applied to any type of process or productto assess performance
For a process: Time required to complete process stepsTimely execution of processNumber of errors in different process steps
Percent yield of a processFor a product: Errors in a productProduct arrives/shipped on timeNumber of products shipped per month
A Measure Is A Description Of One CharacteristicOf An Object Or Activity
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
5/81
Uses Of Data And Measurement
Relation To The DMAIC Methodology
Collect facts about a problem or opportunity and voiceof the customer information
Establish a baseline performance for Project Y to
understand how well we meet customer expectationsIdentify the root cause of a problem and find the key tosolving the problem
Evaluate competing solutions based on their impact onperformance; degree and direction of change; to compareprocess performance before and after the solution isimplemented
Quantify the change in process performance to ensureimprovement gains are sustained
Define
Measure
Control
Improve
Analyze
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
6/81
Process Context For Measurement
CSuppliers Inputs Process Outputs Customers
CTQ CTQ
Input and process measures quantify someaspect of an input to the process or theperformance at one or more process steps. Theyare often referred to as independent variables .
Output measures typically quantify product orservice characteristics or process outcomes.They are often referred to as dependentvariables .
SSupplier CTQs
IInput PProcess OOutput CCustomer S t o p S t a r t
Output Measures Should Reflect What The Customer Feels
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
7/81
The Relationship Between Process Ys And Xs
Subprocess Ys May Be High-Level Xs
SubprocessL2
CoreProcess
L1
Credit Documentation FundingY YY
x = cycle time x = cycle time x = cycle time
Cash Delivery Process
Total Cycle Time = Y
NotificationX
Decision
X
Application
X
Y = Cycle Time
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
8/81
Effectiveness:The degree to which customer CTQs are met and exceeded
Some examples:Percent defectiveResponse time
Efficiency:
The amount of resources allocated in meeting and
exceeding customer CTQsSome examples:
Cost per transactionTurnaround time
Quality Measurement
Time per activity Amount of rework
Billing accuracy
Two Aspects Of Measuring Performance
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
9/81
Why Is Type Of Data Important?
Choice of data display and analysis tools Amount of data required: continuous data oftenrequires a smaller sample size than discrete data
Information about current and historical processperformance
Use Continuous Data Whenever Possible
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
10/81
Types Of Data
Discrete Data
Binary (Yes/No, Defect/No Defect)
Ordered categories (1-5)
Counts
Continuous Data
Can be broken down into increments
Infinite number of possible values
Examples
Number of incomplete applicationsPercent of responding with a 5 onsurveyNumber of Green Belts trained
Examples
Cycle time (measured in days, hours, minutes,etc.)Weight (measured in tons, pounds, etc.)
Data Type Is An Important Consideration
Discrete
C y cl eT i m
e
C o un
t D
a t a
( M an
y p o s si b i l i t i e
s )
Or d
er e
d C a t e g or i e
s
( M an
y o p t i on
s ,i . e. ,1 -1
0 0 )
Continuous
Bi n
ar y
( Y / N )
Technically Discrete, but can oftenbe analyzed as Continuous
C o un
t D
a t a
( L i mi t e
d p o s si b i l i t i e
s )
Or d
er e
d C a t e g or i e
s
( L i mi t e
d o p t i on
s ,i . e. ,1 -1
5 )
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
11/81
Importance Of Data Type
Project Y Discrete Y Measure Continuous Y Measure
Time to process % within specifications Actual times for each unit
Delivery time Number late Actual time deviated from target
Customer satisfaction Yes/no questions Rating 1-100
Policies lost due to price Number lost Difference from competition
The More Continuous We Can Make The Data,The More It Will Tell Us About Our Process
Sometimes we have choices. When we do, we shouldchoose continuous data
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
12/81
Plan For Data Collection
Ensure DataConsistencyAnd Stability
EstablishData CollectionGoals
DevelopOperationalDefinitions AndProcedures
Clarify purpose ofdata collection
Identify what datato collect
Test and validatemeasurementsystems
Write and pilotoperationaldefinitions
Develop andpilot datacollection forms
and proceduresEstablish asampling plan
Collect DataAnd MonitorConsistency
Train datacollectors
Pilot process andmakeadjustments
Collect data
Monitor dataaccuracy andconsistency
1 432
Data Collection Is The First Step To UnderstandingThe Variation The Customer Feels
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
13/81
What To Collect?
Operational Definitions & Procedures
Data CollectionForm
Date # Start StopInstructions
Start:
Stop:Measure Data Type UnitsCycle Time C Days
Sampling Plan
How Many Who
When Where
Training Plan For Collectors
Needs
Form Use
Audit Plan
Analysis Plan
Measure MSA% Error ImprovementSources Plan
MSA / Gage R & R
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
14/81
Establish Data Collection Goals
In order to establish your data collection goals you must:
State the purpose of the data collection
Identify what data is required
Asking these questions may help you clarify your goals:
What do I need to know about my process?
What data do I need?
What is the plan for analysis once the data is collected?
What data is already available?
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
15/81
Segmentation
Collect output data (Y)
To identify patterns and performance trends
Collect segmentation factor data
To be used for later analysis
Segmentation Helps Us Understand Variation In Project Y
Preparing for the Analyze Phase
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
16/81
Step 1: Establish Data CollectionGoals
Segmenting by external factors will help us identifythe drivers of variation in the process
Possible categories:
Product Customer Market Time Geography
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
17/81
Common Factors Used ForSegmentation
Factor Example
What type Complaints, defects, problems
When Year, month, week, day
Where Country, region, city, work site
Who Business, department, individual,customer type, market segment
Tip: Begin with factors outside the process box - often these are factorsthat were not considered when the process was first designed
Other Categories
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
18/81
How To Collect Data ForSegmentation
Identify the factors for segmentation before you startcollecting dataMake sure the segmentation factors can be measuredreliably
Record the segmentation factors for each Y data pointcollectedSegmentation factors are typically easy to collect, socollect more segmentation factors rather than less
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
19/81
Step 1: Establish Data CollectionGoals (5 Minutes)
Table Team Exercise
For your team project:Brainstorm a list of segmentation factors for yourprojectRemember to also segment on unlikelyparametersReport out to group
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
20/81
Develop Operational DefinitionAnd Procedures
Clearly specify variables to be collected:Operational definitions for all metricsSpecific descriptions of how to take themeasurement
Specify the details of the data collection process:How to collect the dataHow to record the dataThe period of time for data collection
The sampling plan to be followed
EstablishData CollectionGoals
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
21/81
Defining The Measure
Purpose
An operational definition is a clear, concisedescription of a measurement and the process bywhich it is to be collected.
To remove ambiguity Everyone has a consistent understanding
To provide a clear way to measure the characteristic Identifies what to measure
Identifies how to measure it Makes sure that no matter who doesthe measuring, the results are consistent
Definition
Always Pilot Your Operational Definitions
Operational Definitions
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
22/81
Features
Operational Definitions For OutputMeasures
What: Must have specific and concrete criteria How: Must have a method to measure criteria Must be useful to both you and the customer
(the wing -to- wing concept)
Example: Loan Application Cycle Time
What: Loan application cycle time is the number of hours fromreceipt of a loan application, to successful notification ofdecision for the loan application
How: The clock starts when the computer attaches the time ofapplication receipt at data entry
The clock stops when the phone caller notes timeof completed application decision notification in desk log
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
23/81
Operational Definitions Scale OfScrutiny
Measure one scale or level smaller than what your customermeasures
For Example:
If your customer measures cycle time in days, your scale ofscrutiny would be hours
If your customer measures cycle time in hours, your scaleof scrutiny would be in minutes
Scale of scrutiny may expose larger true variation
Choo s ing The Level Of Measurement
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
24/81
Operational Definition PartnerExercise (25 Minutes)
What How Who Time
AllPartner Preparation
Develop AnOperationalDefinition
Measure AndRecord Data
Close Exercise
Develop an operational definition for oneof the defect types found in an M&M,either, 1) chips and cracks, or 2) unclear and illegible M.
The definition should include: What How Importance to customer
Find a partner for the exercise.
Determine timing for each activity below.
Read the background information.
With your partner apply your operational definition to your package of M&Ms .
Use the form on the following pageto record the total number of M&Ms you inspect and thenumber of defective M&Ms .
Note: If an M&M has one or m orechips/cracks, classify the M&Ms as defective.
Brainstorm the challenges of developingan operational definition for this exercise,and how these challenges may impactyour own project work.
Choose a spokesperson to report out onyour operational definition, the challengesyou experienced, and how these may
impact your project work in the future.
Partners
Partners
Partners
Desired OutcomesPractice applyingoperationaldefinitionsCollect data on the
number of defects ina package ofM&Ms
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
25/81
Operational Definition PartnerExercise (continued)
BackgroundCustomers of M&Ms candy have various needs related to the consumption of the candy.Because the candy should melt in your mouth, not in your hands, one of the Project YCTQs is for the candy to have no chips or cracks.
Part of the internal process for making the candy is printing the letter M on the candy.While not a high priority for external customers it is important to internal customers formarketing and product branding.
Customer Need
No chips or cracks
Project YCore Process M&Ms Production
Subprocesses
1 2 k
Business Need
Clear and legible Mon the M&Ms
MM
A defective M&M is . . .
1. Any M&M with a chip or crack.2. Any M&M with an unclear or
illegible M.
The two defect types should bemeasured separately.
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
26/81
Operational Definition PartnerExercise (continued)
Data Collection Check Sheet
Date: Location:
Data CollectorsName
# Of PiecesInspected
# Of Pieces ChippedOr Cracked
# Of Pieces WithUnclear Or Illegible M
Data Summary Sheet
Data CollectorsName
# Of PiecesInspected
% Of PiecesChipped
Or Cracked
% Of Pieces WithUnclear Or
Illegible M
% Of PiecesDefective
Operational Definition:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
27/81
Operational Definition TableTeam (5 Minutes)
Write an operational definition for your Project Y Write the definition on a flip chart
Report out
Define Your Project Y
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
28/81
Understand the purpose and advantages of sampling
Understand the application of different samplingtechniques to ensure accurate process representation
Gain experience in asking appropriate questions toensure a robust sampling plan is implementedeffectively and efficiently
Understand guidelines and formulas used to
determine sample size
Objectives
Sampling
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
29/81
Basic Definitions And Symbols
Population (N): The entire set of objects or activities for aprocess
: the mean (arithmetic average) calculated for a
population : the standard deviation calculated for a population
Sample (n): a group that is a part or subset of a populationx: the mean (arithmetic average) of a sample
s: the standard deviation of a sample
Sampling
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
30/81
Sampling
Sampling is the process of:Collecting only a portion of the data that is available orcould be available, and drawing conclusions about thetotal population (statistical inference)
Population Sample
xx
x
xxx
x
x
x
xx
x x
x
x
xx
x
xxx
x
N = 5,000 n = 100
From the sample,we infer that theaverage resolutiontime (x) is 1.2 days
What is theaverage resolution
time?
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
31/81
When To Sample?
When to sample Collecting all the data is impractical or too costly Data collection can be a destructive process When measuring a high-volume process
When not to sample A subset of data may not accurately depict the process,
leading to a wrong conclusion (every unit is unique e.g., structured deals)
Statistically Sound Conclusions Can Often Be DrawnFrom A Subset Of The Total Available Data
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
32/81
Goal Of A Useful Sample
Representative Sample: All parts of the target population are represented
(i.e., selected for measurement) equally The customers view is captured
How to guarantee a representative sample: Designing the sampling strategy Understand special characteristics of the population
before sampling
Representat ive Samp les
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
33/81
Important Sampling Concepts
Bias occurs when systematic differences are introducedinto the sample as a result of the selection process
Not representative of the population
Will lead to incorrect conclusions about thepopulation
Bias
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
34/81
Important Sampling Concepts
Convenience sampling: the ones I can reach Systematic sampling: at noon every day
How Is Bias Introduced?
Selection Bias
EnvironmentalBias
Strategic Level Developing The Sampling Plan
Outdated sample: 1996 external survey results
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
35/81
Important Sampling Concepts
How Is Bias Introduced?
Measurement Bias
Gage R&R Issues
Non-Response
Bias
Tactical Level Carrying Out Sampling Plan
Initiated by respondents: only a subset of the
population responds to survey (typically the1s and 5s)
Inconsistent operational definitions
Inconsistent collectors or procedures (assess usingMeasurement Systems Analysis)
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
36/81
Determine Your Sampling Strategy
Population ApproachMake probability statements about the population from the sample
I have 95% confidence that the mean of the population isbetween1.5 and 2.5 seconds
Use sample size formulaProcess Approach
Assess the stability of the population over time Are the shifts, trends, or cycles occurring?Do I take a special or common cause variation approach toprocess improvement?
Use rational subgrouping
Where Are You Standing?
Process Data
Population Data
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
37/81
Determine Your Sampling Strategy
Think about these two questions:1. What should you do if you are standing at process but wish to
use a population approach?2. What should you do if you are standing at population data but
wish to use a process approach?
Where Are You Standing?
Process Data
Population Data
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
38/81
Sampling Strategy: Random Sampling
Population Sample Description
Each unit (X) hasan equal probabilityof being selected ina sample
Popula t ion S tudy
N n
X X XX X XX X X X XX X X X XX X X
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
39/81
Description
Population Study
Sample
L
MMMM
SS
Population
Segments Units
Large
Medium
Small
LLLLL
MMMMMM MMMMMM
S S SSSSSS
S S
Sampling Strategy: Stratified RandomSampling
Randomly samplewithin a stratifiedcategory or groupSample sizes foreach group aregenerally proportionalto the relative size of thegroup
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
40/81
Sampling Strategy: SystematicSampling
Process Sample Description
Process S tudy
Must select sampling frequency
Sample every n th one (e.g., 4thone)
X XX X X X X X X X X
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
41/81
Sampling Strategy: RationalSubgrouping
Process Sampling
X X X X X X X X X X X X X X X X X X X X X X X X
Hour 1 Hour 2 Hour 3Subgroup ofsamples
Process Sample
Description
Sample at point A in the process every Xth hour
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
42/81
Sampling Strategy: Process Study
Key points to considerMonitor process frequently enough to catch it goingfrom good to bad
Better to collect several small samples over differenttimes then one large sample at a single point in timeUnstable process more frequentlyStable process less frequentlyRapid cycle process time more frequentlyLong cycle process time less frequently
Determine Sam pl ing Frequency
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
43/81
Sampling Situations TableExercise (continued)
You are interested in estimating the proportion of callers (within1%) who experience first call resolution. A customer survey willbe used to gather the data. A random sample will be used to selectpotential survey respondents.1. Are there any potential problems with the approach described?
2. What other approaches might be used in this sampling situation?3. What other information is required?
You are interested in improving billing accuracy and have decided tocollect a subgroup sample of 30 bills processed from 4 to 5 p.m., everyday for the next 4 days.1. What sampling scheme(s) is planned?2. Is there a potential to introduce bias using the plan described? If
yes, how would the bias be introduced?3. What other approaches might be used in this sampling situation?
A
B
S li g Sit ti T bl
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
44/81
Sampling Situations TableExercise (continued)
A business wants to estimate the total cycle time for deals. There are three types ofdeals (large, medium and small) and four regional offices (Atlanta, New York, Chicagoand Los Angeles). The business randomly sampled deals from the Atlanta regionaloffice who had data readily available.
1. Are there any potential problems with the approach described?
2. What other approaches might be used in this sampling situation?
3. What other information is required?
.
D
C
An improvement team is interested in improving billing accuracy. They
decided to sample and pull every 20th bill processed over the next 30 days
1. What sampling scheme(s) is planned?2. Is there a potential to introduce bias using the plan described? If yes, how
would the bias be introduced?
3. What other approaches might be used in this sampling situation?
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
45/81
How Do I Determin e Sam ple Size?
Sample Size For Continuous Data
Sample size (n) depends on three things Level of confidence required for the result, How confident
I am that the result represents the true population Level of confidence increases as sample size increases
Precision or accuracy ( ) required in the result, The errorbars or uncertainty in my result
Precision increases as sample size increases Standard deviation of the population (s), How much
variation is in the total data population?
As standard deviation increases, a larger sample size is needed toobtain reliable results
In this equation, 1.96 represents a 95% confidence level
n = 1.96s ( )2
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
46/81
What Is The Perform ance For Del ivery Time?
Sample Size For Continuous Data
Calculate sample size (n) based on:Precision ( )95% confidence Level (1.96)Standard deviation (s)
Y = DeliveryTime
(Days)
NValues
Conclusion:
I know with 95% confidence that the population mean is X +
Calculate average (X)
Population
Sample
X - X + X
n = 1.96s ( )2
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
47/81
Finite Population Correction
1. Calculate sample size (n)
2. If n / N > .05
ORIf n > N
3. Calculate n finiten finite = n / 1+ n / N
If You Have A Fini te Pop ulat ion
Where n = sample size;N = population size
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
48/81
How Do I Determine Sample Size?
Sample Size For Discrete Data
Sample size (n) depends on three things:Level of confidence required for the result, How confident Iam that the result represents the true population
Level of confidence increases as sample size increasesPrecision or accuracy ( ) required in the result, The error bars or uncertainty in my result
Precision increases as sample size increasesEstimated proportion defective of the population (P)
Sample size is maximized at P = 0.5In this equation, 1.96 represents a 95% confidenceinterval
n = P(1-P)1.96 ( )2
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
49/81
Wh at Is Th e Defect Rate (P) Of A Proces s?
Sample Size For Discrete Data
n =1.96
( )2 P (1-P )
Y = ProportionDefective
nValues
Conclusion:
I know with 95% confidence that the populationproportion defective is P +
Calculate ProportionDefective (P)
Population
Sample
Recalculate n* based on the calculated P. If the new required sample size (n*) is more than thenumber of samples taken, take (n*-n) samples and recalculate P base on the full sample size. If it isnot practical to take more samples, then use the actual n and P to recalculate the actual precision ( )
Calculate sample size (n) basedon:
Precision ( )95% confidence Level (2)Estimated proportion defective (P)
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
50/81
Beyo nd The Form ulas . . .
Sample Size Considerations
The formulas give an approximate sample size Dont forget these important factors!
Is the population homogeneous?If not, you will need to segment before sampling
What is the opportunity segment for bias?Plan ahead to make sure your data is representativeof the true population
What Is The Impact On The Customer If YourSample Size Is Not Representative Of The Process?
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
51/81
Variation In Measurement Systems
Address Measurement System Variation Before
Collecting Data To Analyze Process Variation
Actual ProcessVariation
Observed Variation In Data
MeasurementSystem
Variation
Long Term Short Term Variation In TheMeasurement Tool
Repeatability
Reproducibility
Accuracy
Stability
Linearity
Variation In The Act OfMeasuring
Random
Bias
GageR&R
Issues
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
52/81
Two Cases Of Measurement Error
+ =
+ =
True
Process
Random Variation
Due ToMeasurement
Total Process
Variation Observed
TrueProcess
Random VariationDue To Bias
Total ProcessVariationObserved
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
53/81
Measurement System Analysis (MSA)
MSA is a set of methods for estimating the currentamount of variation in the measurement process
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
54/81
Plan For Consistency And Stability
Data is only as good as the process that measures itIdentifies how much variation is present in the measurementprocessUnderstanding measurement variation is necessary foridentifying true process variation and maximizing true YimprovementsWithout MSA, you run the risk of making decisions based onan inaccurate picture of your process
MSA helps direct efforts aimed at decreasing measurementvariationExcessive measurement variation distorts our understandingof what the customer feels
Why MSA Is Impo rtant
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
55/81
Measurement Systems Analysis
How much variation is caused by the measurement system?Determine which MSA is appropriate based on the type of data collectedDetermine which aspects of measurement are most relevant for the MSA study(accuracy, repeatability, reproducibility, stability, linearity)Measure units repeatedly. How items are measured depends on the aspectbeing quantifiedQuantify the measurement process variation
How much error or uncertainty is allowable for this data?Determine if the measurement process must be improved
What are the sources of measurement error?Determine how the measurement process will be improved
How can the error sources be eliminated or minimizedDetermine how the measurement process will be improvedAudit the measure process to ensure accurate and consistent measurement
Key Ques t ions And Procedures To Answ er Them1.
2.
3.
4.
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
56/81
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
57/81
Measurement System Analysis
The type of Measurement System Analysis conducteddepends on the type of data:
When using continuous data, MSA is conductedthrough a Gage R&R (repeatability andreproducibility) studyWhen using discrete data (discrete, count or orderedcategories), MSA is conducted through a DDA(discrete data analysis) study
Step 1: Determine Which MSA Tool Is Appropriate
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
58/81
Step 2: Determine Which Aspects Of Measurement Error Are Relevant For
Your MSA Study
Measurement System Analysis
Accuracy the differences between observed averagemeasurement and a standard
Repeatability variation when one person repeatedlymeasures the same unit with the same measuring equipment
Reproducibility variation when two or more peoplemeasure the same unit with the same measuring equipment
Stability variation obtained when the same personmeasures the same unit with the same equipment over anextended period of time
Linearity the consistency of the measurement systemacross the entire range of the measurement system
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
59/81
StandardValue
ObservedAverage
Accuracy
Measurement System Analysis:Accuracy
The difference between observedaverage measurement and a master or standard
Continuous Difference between observed and standard inmeasurement units
Discrete Number of instances where the wrong answerwas observed
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
60/81
Measurement System Analysis:Accuracy (continued)
Validating accuracy involves repeated measurement ofsomething with a known value (master/standard) The difference between the average of repeated
measurementsof the same master/standard and the true value of
the master/standard represents the amount ofinaccuracy or bias in the measurement system Service application: Validating the judgement of the
person making the measurement against an agreed-upon master/standard
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
61/81
Step 3: Measure Un its Repeatedly
Word Spell Checker 1 Standard DictionaryCommittee C CBattallion C I
Asbestos C CSeperate I I
Flamboyant I C Abbacus I ICatagory I I
Lieutenant C COccassionally I I
Liquefy C C
Discrete Data Example
A marketer wants to understand the accuracy of his measurement process to measure the number of misspelled wordsin the first draft of a marketing brochure. A spell checker is given a brochure and asked to identify the words that arespelled incorrectly. Any difference between the words the spell checker identifies as misspelled, that are in realitycorrectly spelled (as defined by the standard dictionary), represents the accuracy of the measurement process.
C = Spell Checker identified the word as correctly spelledI = Spell Checker identified the word as incorrectly spelled
= Differences between Spell Checker and Standard Dictionary
DataSummary
Measurement System Accuracy = [(4 + 4)/(4 + 1 + 1 + 4) ] x 100 = 80%
Correct Incorrect
Correct 4 1
Incorrect 1 4
StandardDictionary
Spell Checker 1
5
5
10
5 5
Step 4: Quantify The Measurem ent Process Variat ion
Measurement System Analysis:Accuracy (continued)
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
62/81
Step 3: Measure Un its Repeatedly
20100
9
8
7
6
5
4
3
2
1
Standard Deal #
C y c l e
T i m e
( i n
d a y s
)
Measure of Deal Cycle Time
X=5.063
3.0SL=8.526
-3.0SL=1.599
(20 "Standard Deals")
Continuous Data ExampleA deal business is measuring the cycle time on deals. A reference set of 20 deals (standard) is collectedand a panel of experts determines the true cycle time for each deal. The average cycle time for the 20deals, as measured by the panel of experts is 5.5 days. The normal measurement process thenmeasures the cycle time for the 20 deals (see chart below).
Step 4: Quantify The Measurement Process Variation
The difference between the average of the 20 deals measured by the panel of experts (the standard) and theaverage of the normal measurement process represents the bias, or inaccuracy, of the measurement process.Thus bias is X standard Xnormal , or 5.5 5.1 = 0.4. This means the measurement process has a bias or inaccuracy of0.4 of a day in measuring cycle time.
Measurement System Analysis:Accuracy (continued)
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
63/81
(Minimum Variation)
Repeatability
Measurement System Analysis:Repeatability
The variation when one personrepeatedly measures the same unit withthe samemeasuring equipment
Continuous Calculate variation in terms of measurement units(standard deviation, span, etc.)
Discrete Count number of times the same result is achieved fora given point (% correct)
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
64/81
Measurement System Analysis:Repeatability (continued)
Validating repeatability involves repeatedmeasurement of the same item by one person with thesame measurement deviceThe difference between the first time an item ismeasured and the second time represents the error ofthe measurement processTherefore repeatability is the ability of the person ormeasurement device to consistently repeatmeasurements for the same items
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
65/81
Measurement System Analysis:Repeatability (continued)Discrete Data Example
Data Summary
Word First Check Second Check Committee C CBattallion I I
Asbestos C CSeperate I I
Flamboyant C C Abbacus I ICatagory I I
Lieutenant C COccassionally I C
Liquefy C C
A marketer wants to understand the repeatability of his measurement process tomeasure the number of misspelled words in the first draft of a marketing brochure. Aspell checker is given a brochure and asked to identify the words that are spelledincorrectly. After a short period, the spell checker is given the brochure and asked torepeat the process. The difference between the first and second spell check representsthe spell checkers repeatability.
C = Spell Checker identified the word as correctly spelled
I = Spell Checker identified the word as incorrectly spelled
= Differences between Spell Checks
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
66/81
Measurement System Analysis:Repeatability (continued)Cont inuo us Data Example
A deal business is measuring the cycle time on deals. A reference set of 20 deals is collected andmeasured by one person. The same person then re-measures the 20 deals. The difference between thefirst set of measurements and the second represents the repeatability of the measurement process.
1st 2nd5.4 5.54.7 4.65.5 5.56.3 6.33.9 3.94.7 4.54.8 4.85.5 5.54.6 4.64.7 4.76.2 6.25.0 4.83.9 3.96.9 6.84.4 4.5
4.0 4.04.8 4.87.1 7.23.7 3.75.2 5.4
Source % ContributionTotal Gage R&R 0.43Repeatability 0.43Part-to-Part 99.57Total Variation 100.00
Using either Minitab or Excel we can calculate the percent of the TotalVariation contributed by the measurement process in terms ofrepeatability (see partial printout above).
In this case we see that the repeatability of the measurement process is99.57%, which means that nearly all of the observed variation is comingfrom the process, not the measurement system.
Measurement SystemVariation
Actual Deal Variation
Total Observed Variation+
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
67/81
Reproducibility
Data Collector 1
Data Collector 2
Measurement System Analysis:Reproducibility
The variation when two or more peoplemeasure the same unit with the samemeasuring equipment
Continuous Calculate the difference between two people in terms ofmeasurement units
Discrete Calculate the difference in number of times each personachieved a given result (% difference)
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
68/81
Measurement System Analysis:Reproducibility (continued)
Validating reproducibility involves repeatedmeasurement of the same item by two people usingthe same measurement device
The difference between the two measures representsthe ability of the measurement process to bereproducible
Therefore reproducibility is the ability of themeasurement process to consistently reproducemeasurements for the same items across people
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
69/81
Measurement System Analysis:Reproducibility (continued)Discrete Data Example
Word Spell Checker 1 Spell Checker 2Committee C CBattallion C I
Asbestos C CSeperate I I
Flamboyant I C Abbacus I ICatagory I I
Lieutenant C COccassionally I C
Liquefy C C
A marketer wants to understand the reproducibility of his measurement process to measure the number of misspelled words in the firstdraft of a brochure. Two people called spell checkers are given a list of words and asked to identify the words that are sp elled incorrectly.Any difference between the words the two spell checkers identify as misspelled represents the reproducibility of the measurement process.
C = Spell Checker identified the word as correctly spelled
I = Spell Checker identified the word as incorrectly spelled
= Differences between Spell Checkers
Spell Checker 1
Correct Incorrect
Correct 4 2
Incorrect 1 3
Measurement System Reproducibility = [(5 + 4)/(5 + 1 + 0 + 4)] x 100 = 90%
Spell Checker 2
6
4
5 5 10
5 1
0 4
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
70/81
y yReproducibility (continued)Cont inuo us Data Example
A deal business is measuring the cycle time on deals. A reference set of 5 deals is collected
and measured by two people, and then one of the same people re-measures the 5 deals.The difference between people represents the reproducibility, while the differencebetween the first and second measure by the same person represents repeatability.
Source % Contribution
Total Gage R&R 1.48Repeatability 0.23Reproducibility 1.25Part-To-Part 98.52Total Variation 100.00
Using either Minitab orExcel we can calculate thepercent of the TotalVariation contributed by themeasurement process interms of reproducibility andrepeatability (see partialprintout below).
In this case we see that thereproducibility andrepeatability of themeasurement process is98.52%, which means thatnearly all of the observed
variation is coming from theprocess, not themeasurement system.
Cycle Time Deal # Measurer #5.4 1 15.4 1 15.5 2 15.5 2 13.9 3 13.9 3 14.8 4 14.7 4 1
4.7 5 14.7 5 15.3 1 25.3 1 25.5 2 25.5 2 23.8 3 23.9 3 24.7 4 24.7 4 24.5 5 24.5 5 2
Cycle Time Deal # Measurer #5.4 1 15.4 1 15.5 2 15.5 2 13.9 3 13.9 3 14.8 4 14.7 4 1
4.7 5 14.7 5 15.3 1 25.3 1 25.5 2 25.5 2 23.8 3 23.9 3 24.7 4 24.7 4 24.5 5 24.5 5 2
Cycle Time Deal # Measurer #5.4 1 15.4 1 15.5 2 15.5 2 13.9 3 13.9 3 14.8 4 14.7 4 1
4.7 5 14.7 5 15.3 1 25.3 1 25.5 2 25.5 2 23.8 3 23.9 3 24.7 4 24.7 4 24.5 5 24.5 5 2
Cycle Time Deal # Measurer #5.4 1 15.4 1 15.5 2 15.5 2 13.9 3 13.9 3 14.8 4 14.7 4 1
4.7 5 14.7 5 15.3 1 25.3 1 25.5 2 25.5 2 23.8 3 23.9 3 24.7 4 24.7 4 24.5 5 24.5 5 2
Variation due to measurement system
Actual deal variation
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
71/81
Stability
Time 1
Time 2
Measurement System Analysis:Stability
The variation obtained whenthe same person measuresthe same unit with the sameequipment over an extendedperiod of time
Continuous Calculate variation change in measurement over time
Discrete Calculate change over time (% correct or consistent)
Measurement System Analysis:
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
72/81
y yLinearity
Linearity is the consistency of the measurement systemacross the entire range of the measurement scale
Continuous: The endpoints of a pressure gageare typically not as accurateas the center of the gages range
Discrete: Assessing items for defects is easy in veryobvious cases, but can be very
difficult in borderline or less clear cases. Operators
may have very consistent judgement performance at thebeginning of a shift, but poor consistency before breaksor near the end of a work shift
C d ti A MSA
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
73/81
Step 5: Determine If The Measurement System Must Be Improved
Conducting An MSA
Examine context of business environment,process, and customerHow critical is the measurement?
What are the risks of making an error?
Review results of MSA studyTypical Gage R&R specs:
% < 30% of total processvariation
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
74/81
Conducting An MSA
Identify factors that could causemeasurement process variation(measurement error)
Reduce the impact of those factors
Step 6: Determine How The Measurement Process Will Be Improved
C d ti g A MSA
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
75/81
Conducting An MSA
Determine how often (frequency of audits)Determine what to auditProcedures/documentation up-to-date
Procedures/documentation usedQuantify MSA
Step 7: Audit The Measure Process To Ensure Accurate AndConsistent Measurements
Meas rement S stems
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
76/81
Measurement Systems
Measurement error is always presentin your total observed variation
Minimize the measurement processvariation
Use MSA to identify the amount ofprocess variation
Measurement error is always a biggerdeal than you think
Understand how measurement errorimpacts your customer
Summary
Make Sure Your MSA Is ExaminingThe Actual Measurement System Itself
Measurement Systems Analysis
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
77/81
y yBreakout (35 Minutes)
What How Who Time
AllTeamPreparation
Part 1: MSA For Your Project
Develop an MSA for your project Y data
1. Determine which MSA is appropriatefor your data
2. Determine which aspect of measurement are relevant (accuracy,repeatability, etc.)
3. Develop the plan for how you willcollect the data
4. List factors that might cause themeasurement of an item to vary andhow you would reduce the impact of those factors
OR
Choose facilitator, timekeeper, scribe
All
All
1 min
Part 2: If You DoNot Have AProject
1 min
34 min
5 min
15 min
13 min
How would you run a MSA on thefollowing conditions:
1. Collecting dates from insurance files
2. Errors on billing statements
3. The number of customerscontacted/converted by a broker
4. Call-center: categorizing call types
EstablishData CollectionGoals
DevelopOperationalDefinitions And
Ensure DataConsistencyAnd Stability
Collect DataAnd MonitorConsistency
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
78/81
Collect Data And Monitor
Consistency Communicate the what and why to the data collectors
and process participants Train everyone who will be collecting data Pilot the data collection process and adjust as needed Confirm understanding of operational definitions Make data collection procedures error-proof Be there in the beginning to oversee data collection
Procedures
Collect Data And Monitor
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
79/81
Consistency
Check to make sure the measurementsystem is stable Check to make sure the measurement
proceduresremain consistent (over time, and from datacollector todata collector)
Check to see if the data look reasonable
Summary of Measure
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
80/81
yPerformance
2.1 Determine what to measureUnderstand the role that data plays in process improvementUnderstand the cause and effect relationships that occur inside theteam's processDetermine the indicators needed to evaluate current processperformance
2.2 Manage measurementUnderstand different types of data and how each type can provide theteam with different insights and knowledge of a processDevelop operational definitions and data collection plans that buildvalidity and consistency in the data which the team gathers
2.3 Understand variationUnderstand the concept of variation and how a process can be evaluatedby assessing its variation over timePlot and calculate the variation of the team's business processGain hands-on experience with the use of the statistical softwarepackage MINITAB
ASU Lean Six Sigma Green Belt DMAIC
-
8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)
81/81
Summary of Measure Performance (contd)
2.4 Determine Sigma performanceUnderstand the various calculations associated with determiningprocess sigmaCalculate the sigma performance of the team's processUnderstand the difference between First Pass Yield and RolledThroughput Yield
2.5 Managing the measurement systemUnderstand the different uses of our measurement systemsUnderstand the language of measurementUnderstand how to conduct a measurement system analysisUnderstand how to interpret the results of a MSA study
Understand MSA in administrative processes