10 Steps for Developing Safety Performance Measures

Indiana Safety & Heath Conference

February 28, 2017

Barry S. Spurlock, Esq, CSP

Earl H. Blair, EdD, CSP

Blair & Spurlock, LLC

Step 1. Evaluate, Inventory and Prioritize

• Trends

• Critical Safety Initiatives

• High Impact Drivers

2

Tools for Identifying and PrioritizingPotential Measures

Perception Surveys

Risk Assessment

Injury/Loss Data Analysis

Trending/Benchmarking

Pareto Principle

3

Pareto – The 80/20 Rule

00.51

1.52

2.53

3.54

4.55

Failure toWear PPE

Failure toUse LiftAssist

Lack of PMon LiftAssist

Walking onPallets

Failure toSecureProduct

#/Rate of Incidents

#/Rate of Incidents

Pareto – The 80/20 Rule

33.325.9

20.7

11.1 9

0

10

20

30

40

50

60

70

80

90

100

Walking onPallets

Lack of PM onAssists

Failure to UseLift Assist

Failure to WearPPE

Failure toSecure Product

% of Injuries/Rate By Cause

• Ensure Efficacy of Existing Controls

• Implement Needed Controls

• Identify System and Cultural Blocks

64

Step 2: Verify Efficacy of Hazard Controls ID Obstacles

“Before becoming a KPI, a performance measure needs to be tested

to ensure that it creates the desired behavioral outcome…” [emphasis added]

Parmenter, David. Key Performance Indicators; Developing , Implementing, and Using Winning KPIs, 3rd Ed. , 2015,  Wiley, New York, 2015. 

Key Questions

• Does the intervention/hazard control really work?

• Does measuring the control drive superior performance?

• Is the measure a focal point?

JHA/JSA/THA/Pre Task Planning, 9

Inspections, 9

Training, 8

Near Miss/Near Hit, 4

Management Support/Engagement

, 4

Audits, 4

Safety Meetings, 3

Preventive Maintenance Activities, 3

SOPs/Manuals/Policies, 2

Safety Communications, 2PPE, 2

Management Led Stretching Prgm/ Stretching, 2

Employee Accountability, 2

Most Common Drivers of Safety Performance ‐ 2015  ASSE SeminarFest

Tools

• Pareto Analysis

• Perception/Climate Surveys

• Performance Ratios

• SPC

• Correlation

Training Evaluation Instruments

Level One – Reaction

Level Two ‐ Learning Results

Level Three – Application

Level Four – Business Impact & Return on Investment

adapted from Phillips & Stone

Training Evaluation

66

Value

Level of Training Evaluation Sophistication

Measuring Knowledge Change 

67

KC = Ka

Kb

KC = Knowledge Change

Ka =  Knowledge after training

Kb =  Knowledge before training

Evaluating Training: Knowledge

Knowledge Transfer Ratio = 

Post Training Score – Pre Training Test Score

Possible Score – Pre Test Score

68

SC = Sa

Sb

SC = Observable change in skill as a result of training

Sa =  Skill demonstration after training; output / critical items

Sb =  Skill before training based on same criteria as Sa Fitz‐enz

Measuring Skill Change 

69

AC = Aa

Ab

AC = Attitude Change

Aa =  Attitude after training

Ab =  Attitude before training Fitz‐enz

Measuring Attitude Change 

70

PC = Pa

Pb

PC = Performance Change

Pa =  Performance after training

Pb =  Performance before training

Fitz‐enz

Measuring Performance Change 

SPC

Monthly Accident Control Chart

0

10

20

30

40

Month

# o

f A

ccid

ents

Series1 18 23 14 17 21 33 20 25 22 12 12 10

J F M A M J J A S O N D

UCL

LCL

mean

Correlations

Correlations – The Numbers

Strong Negative

Strong Positive

No Correlation

‐1 0 +1

Correlations

0

2

4

6

8

10

12

14

16

Qtr 1 Qtr 2 Qtr 3 Qtr 4 Qtr 5 Qtr 6

Safe Behavior

Incidents

Correlations

0

2

4

6

8

10

12

Qtr 1 Qtr 2 Qtr 3 Qtr 4 Qtr 5 Qtr 6 Qtr7 Qtr 8

Safe Behavior

Incidents

59

Step 3: Consider Metric Owners w/in Organization

• Top Management

• Middle Management

• Front Line Supervisor

• Employee

60Petersen

Organizational Level

Employees

MICRO

Front Line Supervisors

macro / MICRO

Middle management

MACRO / micro

Top Management

MACRO

Mor

e R

esu

lts

Ori

ente

dM

ore Activity O

riented

60Petersen

Organizational Level

Employees

MICRO

Front Line Supervisors

macro / MICRO

Middle management

MACRO / micro

Top Management

MACRO

Need

for Eng

agem

ent

Need

for Data C

ollection In

tegrity

60Petersen

Organizational Level

Employees

MICRO

Front Line Supervisors

macro / MICRO

Middle management

MACRO / micro

Top Management

MACRO

Ab

ility

to

Ach

ieve

Sta

tist

ical

Rel

iab

ility M

ore Timely Feed

back

Level of Org. Chart

Micro orMacro Focus

Activity or Outcome

Ability for Statistical Accuracy

Need for Data Integrity

Need for Employee Engagement

Need for Timely Feedback

Top Mgmt. Macro Outcome High Lower Lower Lower

Middle Mgmt.

Macro w/ some micro

Mix Moderate Moderate Moderate Moderate

Frontline Sup.

Micro w/ some macro

Mix Moderate Moderate Moderate Moderate

Prod. /Rank

Micro Activity Lower High High High

Prioritize Your Most Important Safety Metrics

71

4 Determine Simple List of Measures

Simple Measures Case Study

You Can’t Measure Everything at Once!

72

Jack Toellner

Select and Customize a Short List of Measures

Identify Three to Five Key 

Measures

30

Gulf Platform Case Study:  Hoover‐Diana Project

Specific measures:1. Safety meetings

2. Housekeeping 

3. Barricade performance 

4. JSA 

5. Safety walks

Jack Toellner31

Results (Outcome Measures)

Hoover‐Diana Platform Project:

• 2 Mil. Hrs of Work

• 1 Recordable Injury

32

5. Identify Means for Employee Engagement

33

Large Brewery: SMP MeasuresEmployee Engagement

34

Important Distinctions about SMP

• Scorecards: Employees are “score‐carded” for safety

• Quotas: Employees are given a quota for safety activities, however…

• Choices: Employees have a choice on which safety activities they will participate in (Coors considers this a key factor in their success)

Anne Bevington

35

36

Individual Participation60 pts

• Observation Cards• Job Safety Analyses• JSA Training• JSA Auditing• Safety Meetings• Safety Audits• Maintenance Walkthroughs• Pre-Shift Stretching• IH Sampling Requests• IH Sampling Results • Ergonomic Assessment Requests• Ergonomic Assessment Actions• Project Walkthroughs• Safety Work Orders• Incident Reviews• Safety Visual Aids• Hazard Alerts• Individual Safety Initiative• Housekeeping Audit

37

SMP Results (Trailing Indicators)

• Average SMP Score for plant 83

• 1 Million Hours w/o Lost Time July 2004 and again in March 2005

• LWCIR  2005 < 1

• Total Case 2005 ‐ 2.4

Anne Bevington

38

6. Develop Methods and Tools

• Checklists• Audits• Observations• Surveys• Scorecards• Dashboards• Audit Reports• Inspections

81

82

Safety Work Orders # or % Completion

# or % Completed on Time

Safety Coaching # of Observations

% Safe

Safety Committee Activity # of Meetings

# of Audits

# Completed Projects

Supervisor Activity # or % Training Completed

# or % JSA Completed/ Updated

Housekeeping Cleanliness Score

Maintenance Score

Example Safety Scorecard

Develop Delivery Strategies

Audience Consideration

FrequencyImpact

83

7.7.

8. Set Performance Goals

• Specific

• Measurable

• Actionable

• Realistic

• Timely

• Aim High

• Employee Engagement

84

Alcoa Case Study Demonstrates Meeting Compliance Goals

Measures Critical Processes

100% Compliance Expected

Monitored Regularly with 

Feedback

85

Compliance Case Study:Requires 100% Compliance

LOTO CSE Fall Prot.Mobile Eqpt.

Measures Critical Four Safety Processes

Must have 100% Compliance to be Acceptable

Monitors Regularly and Provides Timely Feedback Paul Woerz

44

Compliance Example: Leading Indicators

Mobile Equipment

Location Comp with Procedure

s

Documented Obs

Latest Results

Comp with Procedure

sDocumented Obs

Latest Results

Comp with

Procedures

Documented Obs

Latest Results

Comp with

Procedures

Documented Obs

Latest Results

Bethlehem Casting No Yes Yes 100% No NoCleveland Works Yes Yes 99% Yes Yes 99.40% Yes Yes 97% Yes Yes 99.30%Dover Operations Yes Yes 100% Yes Yes 100% Yes Yes 100% Yes Yes 100%Hampton Casting Yes Yes 100% Yes Yes 95% Yes Yes 95% Yes Yes 100%La Porte Casting Yes Yes 100% Yes Yes 99% Yes Yes 99% Yes Yes 100%Morristown Casting Yes Yes 100% Yes Yes 88% Yes Yes 90% Yes Yes 100%Turbine Components Yes Yes 100% Yes Yes 90% Yes Yes 93% Yes Yes 87%Whitehall Operations Yes Yes 100% Yes Yes 99.79% Yes Yes 100% Yes Yes 100%Wichita Falls Casting Yes Yes 100% Yes Yes 100% Yes Yes 100% Yes Yes 100%Howmet/Winsted Yes Yes 100% Yes Yes 94% Yes Yes 96% Yes Yes 83%Gennevilliers Casting Yes Yes 100% Yes Yes 100% Yes Yes 100% Yes Yes 100%Georgetown Casting Yes Yes 100% Yes Yes 100% Yes Yes 100% Yes Yes 100%HJL Yes Yes 100% Yes Yes 100% Yes Yes 97% Yes Yes 100%Tempcraft Yes Yes 100% Yes Yes 100% Yes Yes 100% Yes Yes 100%Evron Casting Yes Yes 100% Yes Yes 100% Yes Yes 100% Yes Yes 100%Dives Yes Yes 100% Yes Yes 100% Yes Yes 83% Yes Yes 100%Exeter Operations Yes Yes 100% Yes Yes 100% Yes Yes 100% Yes Yes 100%Laval Casting Yes Yes 100% Yes Yes 100% No No

Fall ProtectionCSE LO/TO

45

9. Monitor Safety Progress

87

10. Adjust and Modify for Continuous Improvement

88

Directions: We would like to conclude with an opportunity for you to draft a blueprint for your organization.  You can ask questions from the Instructors or other Participants.

1. Do you have a safety measure you use at your organization but you are uncertain if it is effective?  Would you like some feedback?

1. Would you like to share a success story regarding a safety measure that drives success that other participants might find helpful?

48

Concluding Activity

Questions

• earl@blairspurlock.com

• barry@blairspurlock.com

– (502) 595‐7227

REFERENCES AND READINGAppendix A

References & Recommended Reading

ANSI Z10 – 2012.  American National Standard for Occupational Health and Safety Management Systems, American National Standards Institute and American Industrial Hygiene Association, Alexandria, VA: AIHA, 2012. (Note: ASSE is now the Secretariat of ANSI Z10 and holds the copyright)

Bevington, Anne M., 2005.  “Safety Management Process – Proactive Safety Metrics that Drive Performance in Manufacturing Facilities.” Proceedings of the American Society of Safety Engineers 2005 Professional Development Conference, New Orleans

Blair, Earl & Geller, E. Scott  Becoming World Class in HSE Management, Occupational Health & Safety, September 2000.

Blair, Earl & O’Toole Michael F., Leading Measures for Safety Performance: One Way to Enhance Your Organization’s Culture, Professional Safety, August, 2010.

Blair, Earl & Seo, D. C. Safety Training: Making the Connection to High Performance , Professional Safety, October, 2007.

Blair, E. H. & Spurlock, Barry S., “Leading Measures for Safety Performance,” Proceedings of the American Society of Safety Engineers 2008 Professional Development Conference, Las Vegas: ASSE, 2008.

CDC & Prevention, 2001.  Guide to Evaluating the Effectiveness of Strategies for Preventing Work Injuries: How to show whether a safety intervention really works, NIOSH & Institute for Work & Health.  Authors: Linda M. Goldenhar, Andrew R. Hale, Lynda S. Robson, Harry S. Shannon 

Cooper, Dominic, 2009.  Behavioral Safety: A Framework for Success, BSMS.

Daniels, Aubrey C. & Daniels, James E. Performance Management: Changing Behavior That Drives Organizational Effectiveness, Performance Management Publications, 2006.

51

References & Recommended Reading

Daniels, Aubrey C., Bringing Out the Best in People: How to Apply the Astonishing Power of Positive Reinforcement,  McGraw‐Hill: 1994

Geller, E. Scott, The Psychology of Safety: How to Improve Behaviors and Attitudes on the Job, Chilton: 1996

FranklinCovey: The 4 Disciplines of Execution, 2006.

Janicak, Christopher A. Safety Metrics: Tools and Techniques for Measuring Safety Performance, Second Edition, Government Institutes, 2010.

Krause, Thomas R., Safety Incentives from a Behavioral Perspective, Ch.21 in Current Issues in Behavior‐Based Safety, BST, 1999

Manuele, Fred A., 2008.  Advanced Safety Management: Focusing on Z10 and Serious Injury Prevention. John Wiley & Sons.

Mathis, Terry L. “5 New Metrics to Transform Safety” Workplace HR & Safety, September 22, 2009.

McSween, Terry E., The Values‐Based Safety Process: Improving Your Safety Culture with Behavior‐Based Safety (2nd

Ed.) John Wiley & Sons: 2003

Niven, Paul R., Balanced Scorecard Step‐By‐Step: Maximizing Performance and Maintaining Results, John Wiley & Sons, 2002.

Parmenter, David, Key Performance Indicators: Developing, Implementing, and Using Winning KPIs, Third Edition, Wiley & Sons, 2015.

Petersen, Dan, 2005.  The Measurement of Safety Performance, American Society of Safety Engineers.

Pfeffer, Jeffrey & Robert Sutton, The Knowing‐Doing Gap: How Smart Companies Turn Knowledge Into Action, Harvard Business School Press, 2000.

52

References & Recommended Reading

Phillips, J.J. & Stone, R. D., How to Measure Training Results: A Practical Guide to Tracking the Six Key Indicators, McGraw‐Hill, 2004. 

Seo, D. C., Torabi, M.R., Blair, E.H. & Ellis, N.T., “A cross‐validation of safety climate scale using confirmatory factor analytic approach,” Journal of Safety Research, Vol. 35, 427 – 445, National Safety Council & Elsevier, 2004.

Spitzer, Dean R., (2007), Transforming Performance Measurement: Rethinking the Way We Measure and Drive Organizational Success, New York, NY: American Management Association.

Spurlock, Barry S. & Wertz, Keith R., Chomp Comp: Take a Bite Out of Your Workers’ Comp, Lighted Path Publications, 2009. 

Stewart, James M., 2002, Managing for World Class Safety, New York: John Wiley & Sons

Swartz, George, 2001.  Job Hazard Analysis: A Guide to Identifying Risks in the Workplace. Government Institutes

Toellner, Jack, “Improving Safety & Health Performance: Identifying and Measuring Leading Indicators,” Professional Safety, September 2001.

Woerz, Paul , Personal Correspondence, 2006.

53

CALCULATIONSAppendix B

55

UCL / LCL Calculations for #s of Events / Samples

• 95% Statistical Significance = 2 std. deviations from mean = 1.96 = normal distribution

• UCL = x + normal dist. (std. deviation of population)

• LCL = x – normal dist. (std. deviation of population)

• X = mean

• Z = normal distribution

• S= Std. Deviation of Population

56

• 95% Statistical Significance = 2 std. deviations from mean = 1.96 = normal distribution

• UCL = p + 1.96[p(1-p)/n]ˆ0.5

• LCL = p - 1.96[p(1-p)/n]ˆ0.5

• p= mean proportion

UCL / LCL Calculations for Proportions / %

57

Standard Deviation = √Σ (v1 – mean)² + (v2-mean)²….

(n -1)

Standard Deviation

* If only a sample of population.

58

Standard Deviation = √Σ (v1 – mean)² + (v2-mean)²….

n

Standard Deviation

* If entire population sampled.

Pearson Correlation Coefficient

Tip: Use a Calculator with stats functions.