Leveraging Designed Experiments for Success · Leveraging Designed Experiments for Success #1: Get...
Transcript of Leveraging Designed Experiments for Success · Leveraging Designed Experiments for Success #1: Get...
Leveraging Designed
Experiments for Success
Scott C. Sterbenz, P.E.
Six Sigma Master Black Belt, Ford Motor Company
Technical Advisor, United States Bowling Congress
Presentation Outline
I. Introduction
A. Ford Motor Company – The One Ford Plan
B. United States Bowling Congress – Governance of the Sport of Bowling
II. Lessons Learned in Successful DOE Application
A. Get Creative with Your Response:
Premature Bulb Failures – Ford Motor Company
B. The Response Needn’t be Continuous Data:
Carpet Quality – Ford Motor Company
C. Interactions Matter:
Carpet Quality – Ford Motor Company
D. Expand the Range of Your Factors:
Static Weight Study – United States Bowling Congress
E. Don’t Forget About Center Points:
Static Weight Study – United States Bowling Congress
III. Questions & Discussion
2
Ford Motor Company
The One Ford Plan:
• Lays a foundation for business success
• Focuses on working together to achieve profitable growth for all
• Facilitates leadership in four pillars for customer satisfaction and value
3
The United States Bowling Congress:
• Vision
Create lifelong bowlers
• Mission
Provide benefits and programs to enhance the bowling experience
Equipment Specifications and Certification Department:
• Vision
Uphold the credibility of bowling
Leading source of technical information
• Mission
Bring science, technology and bowling together
Solve problems, answer questions, and implement specifications
Expert technical services and sound statistical analyses
4
United States Bowling Congress
Leveraging Designed Experiments for Success
#1: Get Creative With Your Response
5
Get Creative With Your Response
Training materials for designed experiments teach that the response should be:
• The KPOV of the process
• Directly related to the customer CTQ
• Selected from the C&E matrix, fishbone diagram, or y=f(x) cascade
Generally, these guidelines are true, but sometimes yield a nonmeaningfulmeasure.
6
Get Creative With Your Response
Practical Problem: Premature Bulb Failures
Background:
• Warranty costs in 2005 were $2.7M, and increasing every model year
• Single highest warranty cost in Ford Motor Company
• Disagreement between vehicle subsystems about root cause
Over-voltage
Vehicle vibration
Supplier quality
7
Get Creative With Your Response
Practical Problem: Premature Bulb Failures
Plan:
• 25 full factorial DOE:
Vibration input
Voltage input
Bulb supplier
Filament orientation angle
Filament centering
• Bench test:
Typical customer usage cycle
Twenty bulbs (replicates)
8
What should the response be?• Average time to failure
• Variance in time to failure
• Signal-to-noise ratio
Is there something better?
Get Creative With Your Response
Practical Problem: Premature Bulb Failures
Selected Responses:• Reliability shape• Reliability scale (B63 Life)
Analysis:
9
Reliability Plots Constructed for
Main Effects and Interactions
Probability Plot of Effects to
Illustrate Significance
Get Creative With Your Response
Practical Problem: Premature Bulb Failures
Results:• Cumulative savings since implementation (2008MY) = $5.6M• Design rules for voltage regulation at incandescent lamps
10
Implementation
Leveraging Designed Experiments for Success
#2: The Response Needn’t Be Continuous Data
11
The Response Needn’t Be Continuous Data
Training materials for designed experiments teach that the response must be continuous data—not attribute data.
This guideline is true, but sometimes measurements are not possible to be continuous.
12
Ordinal or Nominal Data
(NO)
Ratio or Interval Data
(YES)
Practical Problem: Fiesta Carpet Quality
Background:
• Critical vehicle launch for Ford Motor Company
Largest threat to a quality launch
Anticipated customer satisfaction concerns
• Ford and supplier at odds
Competing responses—brush marking and softness
Cost versus quality
Promises versus deliverables
13
The Response Needn’t Be Continuous Data
Brush Marks
Practical Problem: Fiesta Carpet Quality
Plan:
• 26-1 fractional factorial DOE:
Six factors
Two center points
Two replicates
• Evaluations:
Five evaluators
Brush marking and softness
Likert scale
14
The Response Needn’t Be Continuous Data
Individual collected responses are attribute—What can be done?
Leverage replicates / multiple evaluators
• Transforms ordinal Likert scale
Increases resolution from units digit to tenths digit
Mimics continuous data
Practical Problem: Fiesta Carpet Quality
Selected Responses:• Average softness rating• Average brush marking rating
Analysis:
15
Abbreviated DOE Matrix Shows
Transformation of Likert Scale (Ordinal Data)
The Response Needn’t Be Continuous Data
Pareto Chart of Effects Illustrates
Standard DOE Analysis
Leveraging Designed Experiments for Success
#3: Interactions Matter
16
Interactions Matter
Training materials for designed experiments teach that three-way and higher interactions are rare (Sparsity of Effects Principle).
Generally, this is correct. However, there are some cases where three-way interactions are not only present, but also very strong:
• Complex manufacturing processes
• Chemistry
• Psychology
17
Practical Problem: Fiesta Carpet Quality
Selected Responses:• Average softness rating• Average brush marking rating
Analysis:
18
Pareto Chart Shows Significant and
Strong Three-Way Interactions
Interactions Matter
Minitab “Display Available Designs”
Details Resolution of Designs
Practical Problem: Fiesta Carpet Quality
Results:
• Full extent of interactions understood
Fosters technical excellence
Replication of knowledge
• Multi-Response Optimization
Softness and brush marking
19
Interactions Matter
Minitab Optimization Plot Illustrates
Balance of Multiple ResponsesElimination of Brush Marking;
Softness Better Than Baseline
Leveraging Designed Experiments for Success
#4: Expand the Range of Your Factors
20
Expand the Range of Your Factors
Training materials for designed experiments teach that factor levels should:
• Be wide enough to create a desired change in the response
• Go beyond typical limits in the process
• Not create unsafe or impossible conditions
This guideline is absolutely correct:
• Don’t be afraid to make bad parts
• Challenge the limits of the tools
21
Practical Problem: Static Weight Study
Background:
• 2007 study determined factors that affect ball motion on a lane
High Influence – coverstock
Moderate Influence - core
Low Influence – static weights
22
Expand the Range of Your Factors
Practical Problem: Static Weight Study
Background:
• 2007 study evaluated static weights within current specifications
• Bowling ball manufacturers requested removal of specification
• USBC concerned static weights were influential outside current specifications
23
Expand the Range of Your Factors
Definition of Static Weights
Finger
Thumb
Positive
Side
Negative
Side
Top (Grip Side)
Bottom (Not Visible)
(±1 oz.)
(±1 oz.)
(±3 oz.)
Practical Problem: Static Weight Study
Plan:
• 26-1 fractional factorial DOE:
Six factors
Three static weights
Core shape (intermediate diff.)
Ball speed
Rate of revolution
Eight center points
24
Expand the Range of Your Factors
How wide should the levels be?
USBC investigating completely removing static weight specification:
• Static weights set at maximum possible values
• Core shape is significant to ball motion
• Ball speeds and rates of revolution cover all bowling styles
Practical Problem: Static Weight Study
Selected Responses & Analysis:
• 19 measures characterize ball motion
Collected from CATS (computer-aided tracking system)
Transition points between phases
Lengths of the phases
Shape of the phases
25
Expand the Range of Your Factors
23 Lane Sensors
Track Ball Motion
Regression Techniques Used to
Characterize Ball Motion Mathematically
Expand the Range of Your Factors
Practical Problem: Static Weight Study
Results:
• Anomalies discovered before the DOE was analyzed
Residuals analysis from regression
Undesirable 4th phase of ball motion discovered
26
Initial Results from Ball Motion Algorithm
Residuals Analysis in Roll Phase
Shows Missed Quadratic Term Correction Shows 4th Phase of Ball Motion
Expand the Range of Your Factors
Practical Problem: Static Weight Study
Results:• 4th phase is unpredictable
Unfair advantage Athlete dissatisfaction
27
Angle into Pins DiminishedAngle into Pins Augmented
Expand the Range of Your Factors
Practical Problem: Static Weight Study
Results:• 4th phase is unpredictable
Unfair advantage Athlete dissatisfaction
28
Normal / Expected Ball Motion 4th Phase / Unexpected Ball Motion
Leveraging Designed Experiments for Success
#5: Don’t Forget About Center Points
29
Don’t Forget About Center Points
Training materials for designed experiments teach to include center points:
• Increases power of the experiment
• Helps eliminate saturation
• Evaluates linearity of the response
This guideline is absolutely correct:
• Can lead to use of Response Surface design
• More accurate modeling
30
Practical Problem: Static Weight Study
Analysis:• DOE analyzed without responses affected by 4th phase• Curvature was significant in 18 of 19 responses
31
Don’t Forget About Center Points
Typical Result Indicating
Significance of Curvature
Analysis of Variance for A-Score (coded units)
Source DF Seq SS Adj SS Adj MS F P
Main Effects 6 0.0099790 0.00997901 0.00166317 2544.74 0.000
2-Way Interactions 15 0.0005627 0.00056270 0.00003751 57.40 0.000
3-Way Interactions 10 0.0000638 0.00006383 0.00000638 9.77 0.003
Curvature 1 0.0002570 0.00025705 0.00025705 393.30 0.000
Residual Error 7 0.0000046 0.00000458 0.00000065
Pure Error 7 0.0000046 0.00000457 0.00000065
Total 39 0.0108672
Don’t Forget About Center Points
Practical Problem: Static Weight Study
Analysis:
• Central Composite Design
Reduced static weight levels:
o Attempt elimination of 4th phase
o Widen specification, not elimination
32
Don’t Forget About Center Points
Practical Problem: Static Weight Study
Results:
• Non-linear effects confirmed
• 4th phase of ball motion still present; direction and occurrence not predictable
• Effects of static weights within current specifications insignificant
• Static weight specification not changed
33
Contour Plot—Static Weights Within Specs Are InsignificantPareto of Effects—Order of Effects and Non-Linearity
Presentation Summary34
Topic Lessons Learned
Get Creative with Your Response 1. Think beyond mean and standard deviation
The Response Needn’t Be Continuous Data1. Leverage replicates
2. Convert attribute data
Interactions Matter1. Select proper design resolution
2. Improve process optimization
Expand the Range of Your Factors1. Discover what happens outside the typical
inference space
Don’t Forget About Center Points
1. Check linearity assumptions
2. Achieve greater knowledge with response
surface methods
Leveraging Designed Experiments for Success
Questions & Discussion
35