Metrics and the Cost of Metrics and the Cost of QualityQuality

Chapter 18

Why Metrics!

A strategy without metrics is just a wish. And metrics that are not aligned with strategic objectives are a waste of time.– Emery Powell

Be careful what you measure -- you might just get it.

If you don’t keep score, you’re only practicing.– Tom Malone

You get what you inspect, not what you expect.

The Real Bottom Line: Mission and Measures

One of the most powerful management disciplines, the one that more than any other keeps people focused and pulling in the same direction, is to make an organization’s purposes tangible. Managers do this by translating the organization’s mission – what it, particularly, exists to do – into a set of goals and performance measures that make success concrete for everyone. This is the real bottom line for every organization – whether it’s a business or a school or a hospital. Its executives must answer the question, “Given our mission, how is our performance going to be defined?”– Magretta & Stone, Management. 2002, p. 129

Why Metrics Now!

The “never satisfied” customer. Managing the “total” supply chain. Shrinking product life cycles. More (not necessarily better) data. Profit margin squeezes. Presence of an increasing number of

alternatives.

So What

Surviving in this new environment means working with:– Less lead time– Less inventory– Less cost– More reliance on the supply chain

Surviving means having “better” metricsmetrics.

What are Metrics?

A verifiable measure stated in either quantitative (e.g., 95% inventory accuracy) or qualitative (e.g., as evaluated by our customer,we are providing above average service) terms. A metric is intended to close the gap between value, strategy, and specific activities.

Metrics– Measure, direct, teach.

Metrics – The Firm’s Instrument Cluster

Metrics Categories

Metrics– Individual measures.

Metrics Set– Number of metrics used to evaluate the

performance of an individual or unit. Metrics System

– System for integrating, coordinating and alignment metrics, objectives, and plans.

TYPES OF METRICS

Individual vs. Systems Operational vs. Financial

– Utilization vs. Internal Rate of Return (IRR)• Traditional focus - financial

– Critical issue• Operational improvements do not always result in

financial results Output vs. Process Internal to the firm vs. supply chain metrics

Types of Metrics

Product/Output metrics– Measure results, not the performance of the

processes that gave rise to this performance. Process metrics

– Pertains to the performance of the process– Typical measures -- cost, time, quality, agility

(the ability to adapt), capacity, variability

Output vs. Process Metrics

You have to drive to Detroit. You would like to leave by no later than noon. You need to arrive at no later than 3:00 pm.

Output metric - 3 PM arrival time met Process metrics

– departure time - 1:00 PM– average speed - 65 mph

Importance of Process Metrics

Experience is a wonderful thing. It enables you to recognize a mistake when you make it again - and again - and again. Process metrics enable you to recognize a potential mistake before you make it.

Key Point

Output metrics - critical to top management

Process metrics - critical to those working in execution

Metrics Process

I - Intent (what you want to achieve and why?) D - Deployment (how do you want to achieve this goal?) A - Assignment (who will be held accountable?) S - Standard (how will you measure achievement?)

• Issues of form, source, feasibility, appropriateness.

E - Evaluation and Reward

I.D.A.S.E. Process in Detail

Intent– Desired outcome– What is it that we are trying to measure?– Why is it that we are measuring this?– How does this process/outcome support our objectives

at the functional level? At the corporate level?– What would happen if we were to fail on this

dimension?– Who are the “champions” for this outcome?

I.D.A.S.E. Process in Detail

Deployment– How are we going to achieve this outcome?– What type of data do we need to effectively and

accurately implement this metric?– Do we have this data? If not, what proxy data can we

use? Impact of the proxy data?– Will this be implemented as a predictive or outcome

metric?– Impact of the metric on the metric set?– Is this metric similar to any others currently in use?

I.D.A.S.E. Process in Detail

Assignment– Recognize that there are two different people

involved in a metric• Those who monitor the outcomes• Those who manage the processes that create the

outcomes

– There are different behaviors• Scanners -- large number of metrics-- looking for gaps• Searchers - deal with fewer metrics -- interested in

problem solving

I.D.A.S.E. Process in Detail

Standard– The basis of comparison– Defines what is acceptable (and unacceptable)

behavior– Different types of metrics– Can be generated from the benchmarking

process– Can and should be dynamic

Standards

Absolute– EPA– Few in number

Relative– Relative to our performance

• Past performance

• Best possible future performance

• Pros and cons

Standards - 2

Relative– Relative to best within group

• Need to develop appropriate group• Best within the group teach others• Worse within the group learn from others• Group membership must be viewed as dynamic

– Relative to a “benchmark”• Best in firm• Best in industry• Best in class

FACFACCritical Traits of Critical Traits of

StandardsStandards

F - Feasible

A - Appropriate

C - Comparable

An Example of Metrics in Action

OEE– Operational Equipment Effectiveness– Availability * Performance * Quality– Note what we are measuring

• Quality• Uptime• Cost

– Note what we are not measuring• On-time delivery

Metrics Set

The set of measures that we use to monitor or evaluate the performance of a specific person or function or activity.

Metrics set brings together two elements– Set of metrics

• Load

– Capacity to interpret or act on the measures• Capacity

Metrics set recognizes that the process is capacitated.

Metrics Set

Ideal number of measures– 5 +/- 3

If we overload the manager, then we get:– Frustration– Confusion– Satisficing– Negotiation– Cherry picking

Metrics System

Metrics System– Total system responsible for coordinating and

aligning metrics• Alignment

– Top to bottom

• Coordination– Across functions

– Why are Alignment and Coordination so important?

• The lessons of the “telephone” game.

Alignment and Coordination

Alignment occurs first– Coordination follows

Alignment involves a translation problem– Restating higher order objectives into supporting activities

for the lower levels– As we go from top to bottom, objectives narrow and level

of details increase

Challenges– Translation gets “messed” up– Multiple bosses

Attaining Alignment

Approaches– Bill of Material Approach– Balanced Scorecard Approach– Theory of Constraints Approach– Strategic Profit Impact Model

Approaches for Managing the Metrics System

Bill of Material approach– Explosion of metrics from the top down– Converting each higher level outcome into a series of

lower level outcomes that must be achieved to ensure that the higher level outcome is met.

– Problems

Balanced Scorecard– Approach– Problems

Balanced Scorecard Approach

Approaches for Managing the Metrics System

Theory of Constraints Approach– Must look at the entire system and identify the

constraint– Now measure the performance of the constraint– The constraint drives the performance of the:

• Firm• Supply chain

– Constraint must always be working• Measure this

Approaches for Managing the Metrics System

Theory of Constraints - continued– Elements

• Profit (P)

• Operating Expenses (E)

• Throughput Time (T)

• Inventory (I)

Strategic Profit Model

Recent Findings Metrics are a source of fear Different groups perceive metrics

differently– Resolution strategies:

• Negotiation (group optimization)• Satisfaction of minimum targets (order

qualifiers)• Decompose the problem into sub-problems

(silo management)• Treat everything as important (cope as you can)• Find new metrics we can agree upon• Ignore most and focus on what I can do well• Seek divine guidance

Recent Findings Metrics must be managed as a

system Metrics for motivation and those

for process control are different– Global vs. local

Metrics explosion Ceteribus Paribus

Metrics Framework

Output Process

Multidimensional

Focused

Line of Monitoring

Line of Control

Dimensionality

Metric Type

Aggregate Point Control

Motivation Embedded

– Focused on the critical few, not trivial many

– Linked to value– Measure directly– Appropriate display– Appropriate frequency of updating– Acceptable time lags between

generation of metric and report– Appropriate standard used

Traits of Effective Metrics

The “Cost of Quality” Metrics

Types of Quality Costs– Prevention– Appraisal– Internal Failure– External Failure

Classifying Quality Costs

1-10-100 Rule Theory Says:

– Prevention – 1– Inspect and Prevent – 10– Given to Customer – 100

1-10-100 Rule Practice Shows:

– IBM Rochester: A400 Server• 1-13-92 (hours work required)

– Corning Glass: Glass pressings• 1-80-400 ($ cost)

– Texaco: Refinery• 1-10-100 ($ cost)

Cost of Customer Defections– Companies can boost profits 25 –

85% by decreasing defections 5%

Cost at Points in the Process

Use of a Cost Basis

Why?– Allow comparison across different periods

Typical Bases– Internal failure as a percent of total production costs– External failure as an average percent of net sales– Procurement appraisal costs as a percent of total

purchased material costs– Operations appraisal costs as a percent of total

production costs– Total quality costs as a percent of production costs

Quality Cost Basis ExampleJanuary February

Cost Category Product A Product B

Product A Product B

Prevention $2,000 $4,000 $2,000 $4,000

Appraisal $10,000 $20,000 $13,000 $21,000

Internal Failure

$19,000 $106,000 $16,000 $107,000

External Failure

$54,000 $146,000 $52,000 $156,000

Total $85,000 $276,000 $83,000 $288,000

Standard Direct Labor Costs

35,000 90,000 28,000 86,000

Quality Cost Basis Solution

Use of the System

Categorizing costs Estimating true external failure costs Application of cost bases Tracking improvement ROQ