sneed

Edison Electric Institute

(EEI)

Leading Indicator

Industry Practices

Whitepaper

EEI Leading Indicator Project Team

April 30th, 2014

Information Technology Solutions SSaaffeettyy LLeeaaddiinngg IInnddiiccaattoorrss Supporting Employees Along The Road To An Injury Free Workplace

1

Table of Contents

1. Introduction .............................................................................................................. 2 2. Method ..................................................................................................................... 2

3. Trends and Findings ................................................................................................. 2 4. Prospective Solutions ............................................................................................... 4

5. Conclusions and Recommendations ........................................................................ 6 6. References ................................................................................................................ 8 7. Glossary ................................................................................................................... 9 8. Appendix A: Additional Analysis Tables .............................................................. 11

9. Appendix B: Survey Items with Summary of Survey Data ................................... 14

2

1. Introduction There are two functions to an organization’s safety performance: effort and

achievement (Kehoe, Claybaugh, Naglosky, Korczak, & Sauer, 2011). In order to

effectively manage hazards, safety metrics must account for both the effort made and the

results (i.e., achievement). Traditionally, companies have focused on the results or

lagging aspect of safety metrics such as Days Away, Restricted, and Transfers (DART)

and Occupational Safety and Health Administration (OSHA) recordables. However, it is

also necessary to include measures of the effort which lead to the results (i.e., leading

indicators).

This white paper was developed under the direction of the Edison Electric

Institute (EEI) Occupational Safety & Health Committee and is intended to share Leading

Indicator practices used within the utility industry. For years safety professionals from

across the globe have been touting the importance of incorporating the pro-active

approach of identifying, measuring and evaluating the information received from leading

indicators as the best practice to assure a safe work environment. Leading indicators are

proactive inputs or predictors of future safety performance based on selected criteria

(Fitzgerald, 2011). They place focus on the safety process and must be tied to specific

hazards (Kehoe et. al., 2012). When developed properly, leading indicators may have a

direct relationship to performance and thus provide valid mitigation actions. With the

institution of leading indicators, modifications to operations can be made before injuries

occur.

2. Method An electronic survey was sent to the membership of the Edison Electric Institute

(EEI) Occupational Safety & Health Committee members regarding leading indicators.

Approximately half of the membership companies responded, which allowed for a quality

sample size for verification. The survey included 26 questions covering demographics,

leading indicator types and effectiveness. The focus of the study was best practices in the

industry. Data on OSHA, DART, Lost Time, and Severity rate was also collected.

Traditionally, lagging safety indicators are used by companies to measure injury

data after the fact. Programs, policies, and procedures are developed primarily based on

this reactive type of data. However, an evolving trend in the electric utility industry, is

the incorporation of leading indicators as a measure of safety performance so they can

intervene before an accident happens.

3. Trends and Findings The primary outcome variables of interest in the survey were OSHA and DART

rates. Therefore, trends are reported in terms of their apparent relationship to OSHA and

DART rates.

Information was collected simply on whether or not participants track leading

indicators. Companies who track leading indicators had significantly lower OSHA and

DART rates than those who indicated that they do not track leading indicators. Therefore,

it can be reasoned that the mere presence of a leading indicator program, regardless of its

3

parameters, is associated with lower injury rates. Furthermore, participants were asked to

indicate what they believe the effects of their leading indicator program to be.

Approximately half of the companies’ who use leading indicators report that they have

resulted in a meaningful reduction of their DART rate.

Participants were also asked to indicate their communication strategies for leading

indicator metrics. One of the areas of research which was believed to be important and

have an effect on outcomes was how the leading indicator information is shared with

employees. Companies indicated their strategies were as follows:

50% report out to upper management,

66.7% report to all management

60% report the metrics to all employees

The data were assessed to evaluate how the companies in the top quartile of DART

(i.e., the lowest DART rates) communicated their leading indicators; this information was

different than the percentages above. All the top quartile companies had structured their

delivery of leading indicator results. The top quartile companies also communicated to all

of their employees, and not (only) to selected group.

Below is a chart to give the readers additional information on some of the leading

indicators that were communicated by the companies in the top quartile of DART:

Most Frequent Leading

Indicators Communication

Report

Level

Additional Leading

Indicators associated

with better

performance

DART for

Top

Quartile

Companies

1.Safety Observations

2. Training,

3. Improvement plans/

Corrective Action Plans

All employees

Top down –Bottom

up

Executive level to

Operational level to

local area work

centers

BBS, Safety Team

Effectiveness, Near

Misses, Audits, SWOs

Ninety percent of respondents are tracking safety leading indicators and the

majority of companies that use leading indicators as a metric are reporting only the

quantitative data (just the numbers). Although only producing quantitative data does not

appear to be the best form of communication, it is the opinion of the research team (based

on the outcome of the data) that the level of heightened awareness and openness of

communication influences an environment of caring and commitment to improving safety

which ultimately affects a strong safety culture.

4

The old adage of “if it’s not measured, it’s not managed” was very apparent in the

research as the top safety performing companies tracked their leading indicators and

incorporated them into organizational goals and objectives.

Based on the results of the survey, many of the companies indicated that their

attention to focus on leading indicators is in its early stages. Breaking the survey down

we find that:

10% of participants have been measuring leading indicators for less than a year

27% of participants have been measuring leading indicators for 1-3 years

17% of participants have been measuring leading indicators for 3-5 years

23% of participants have been measuring leading indicators for 5-8 years

10% of participants have been measuring leading indicators for 8- 10 years

13% of participants have been measuring leading indicators for 10 or more years

Most of the companies with a more mature leading indicator program were in the

top quartile of the study. The survey respondent’s DART rate ranged from .36 to 2.75

while DART for the 2012 EEI industrywide safety survey was .10 to 6.61.

4. Prospective Solutions This study found that behavioral safety programs were used in ALL top decile

companies. The implementation of Behavior Based Safety (BBS) programs was shown to

be a universal best practice for all top decile companies.

Safety Observations are the most frequently used measure associated with injury

prevention. Top decile companies incorporate Safety Observations into their safety

metrics and measured their results in both quantitative and qualitative formats.

All top decile companies had incorporated software solutions for capturing safety

observation data and training data. Systems and software solutions allow companies to be

consistent in identifying trends throughout their organizations, pro-actively, while

communicating improvement plans throughout their enterprise. Doing so will often times

mitigate a potential injury. An example of this could be as simple as an observation of an

employee using outdated PPE which had been recalled was identified in one area of the

organization but for some reason the initial communication of the recall was never

received by everyone and a re-issue of the recall was part of the improvement plan.

The survey identified that a safety observation program was the number one leading

indicator for injury prevention. Safety Observations are inclusive of BBS metrics. BBS

metrics have a stronger correlation in reducing injury rates indicating that the structure of

a BBS program is what is needed to drive injuries down.

The success of a safety observation program is enhanced by a BBS approach for the

following reasons:

Behavior focused

Interactions required in the field

Recording of observed behavior and interactions

5

Data trending and reporting

Trend focused and preventative

A successful training program which focuses on more than meeting the minimal

compliance requirements and incorporating identified or observed trends was noted by all

top quartile companies as a piece of the puzzle to a successful safety culture. A solid near

miss program and an improvement plan program or corrective action program (CAP)

were also identified as key performance metrics required to support a sustainable safety

culture.

Inclusion of BBS initiatives as a component of performance indication can help build a

case for change resulting in:

Improved injury rates

Reduction in associated Workers Compensation costs

Produces a measured performance across a consistent and common set of metrics

Incorporates rollup status across programs and divisions

Ensures entire organization knows what requirements need to be met

Establishes an enhanced work environment

Creates a new way of working – employee engagement

The goal and objective of any organization and safety professional is to provide

an environment free of recognized hazards. Our data supports the notion that a pro-active

approach to safety pays dividends. As the research team focused on rates, we began to

discuss the impact of a fatality on the rates. Below is an opinion based on the authors’

discussion while developing this paper.

An important fact to share is the impact of a fatality on rates if costs were

normalized. Although not scientific, we believe that sharing our thoughts may influence

upper management’s support of a leading indicator program.

Because the outcome below is based on a small group’s opinion, the authors of

this white paper would like to re-iterate that the information below is opinion and based

on assumption but it was believed that this information was not ever discussed in research

papers and may express that the lagging indicator rates that we capture year in and year

out are potentially not weighted to reflect the true impact to an organization:

Assumptions:

A couple of years ago (2007/2008) the average lost time cost was about $35,000

per NSC.

The BLS indicated 7 or 8 days was average time out per lost time in 2005 or so

and this goes up exponentially (doubles) when you have an aged work force

(which we all do).

The average fatality costs an organization anywhere from 4 million to 9 million

(Waerher et.al., 2004)

Therefore: The cost of 1 fatality equates to the cost of 114 lost time events:

6

Logic:

Using 4 million dollars as the baseline of a fatality and 35 thousand dollars as the

base line for a lost time injury; you could equate one fatality to 114 lost time events

which would significantly change the lagging indicator rate. (4,000,000/35,000)

Statement: A DART rate of 0.5 (25 DART cases) for a 5000 employee company

would increase to 2.78 with 1 fatality based on the above methodology.

Result: From an economic position, the economic impact of a fatality would move a

top decile company’s DART rate to fourth quartile.

5. Conclusions and Recommendations After reviewing all the data received it is the recommendation of the executive

committee of EEI recommends the following:

General recommendations:

Develop a long term plan (begin with a five year plan) that can be implemented,

improved and sustained –

o A strategy of a multi-year implementation plan occurs which begins with

management and safety professionals working together to design the goal

of the program, the manner in which you plan to capture the received

information and the process of engaging the workforce.

Communicate, communicate, and communicate. Design, develop and deliver a

consistent approach to reporting safety. Include safety metrics in an easy to

understand manner and communicate your organization’s successes and

opportunities to improve.

Safety Observations

All companies in the electric industry include a safety observation program which

incorporates measured results.

o Observations should be focused on behavior based safety rather than

conditions

o Results should be measured via an automated system.

Training

Include safety training on an annual basis.

o Engaging the work force to identify training improvements is very

powerful and this has been seen as a common theme within the research.

This training will not only include compliance based training but it will

refresh our expectations associated with leading indicators, safety culture

as well as the identified areas of improvement. This safety training shall

7

include the improvement opportunities and any new safety initiatives such

as an observation program, a near miss program as well as the reasoning

behind organizational decisions to implement a safe work practice which

goes above and beyond the minimal standards which OSHA and other

agencies hold the industry to.

Near Miss

Develop a near miss or close call program and recognize successes. The great

element associated with leading indicators is that OSHA encourages recognizing

pro-activity and actually frowns upon incentive programs associated with injury

reporting (lagging).

The EEI Occupational Safety and Health committee will continue to focus on leading

indicators and committees have been or will be developed to provide guidance on the best

way to introduce and implement the following:

1. Safety Observations

2. Safety Training

3. Near Misses/Close Calls

4. Safety Awards and Recognition

5. Area Audits/Safety Walkthroughs

8

6. References

Waehrer GM, Dong XS, Miller T, Haile E, & Men Y. Costs of occupational injuries in

construction in the United States (2004).

Fitzgerald, Bob, Manager—Construction Safety & Health, Southern Company (2012).

Measuring Leading Indicators. Southern Company, Engineering & Construction

Services

Kehoe, Gary, Claybaugh, Don, Naglosky, Bill, Korczak, Nick, & Sauer, Shane (2012).

Leading Indicator Development. Edison Mission Energy.

9

7. Glossary

Leading Indicators A leading indicator is a measure preceding or indicating a future

event used to drive and measure activities carried out to prevent and control injury.

Lagging Indicators Lagging indicators measure a company’s incidents in the form of

past accident statistics.

Safety Work Orders (SWO) Work orders generated and flagged as Safety

Walkthroughs A thorough assessment of a facility based on

program/procedure/policy requisites or aspects along with resident representatives when

dialogue is engaged regarding observations made and recommendations for corrections

are highlighted. In addition, audited employees are free to share constraints/issues

encountered during routine/non-routine activities.

Behavior Bases Safety (BBS) The use of behavioral psychology to promote safe

behavior in the workplace using employee engagement.

Corrective Action Plan (CAP)/Improvement plans The identification of specific,

meaningful resolutions to hazards which are created as a result of an incident and are

tracked to ensure they are completed on time.

OSHA Occupational Safety & Health Administration

DART Days Away Restrictions & Transfers

p Value (i.e., significance level) Indicates the probability that the results of a given

analysis are due to chance.

Standard Deviation Indicates how tightly all the various cases are clustered around

the mean.

10

R-Squared Describes the percentage of variance in the dependent variable (i.e., OSHA

& DART for the current study) which can be explained by the independent variable (i.e.,

number of leading indicators tracked).

N Sample size (i.e., number of respondents)

x Average

f ratio Statistic derived from an Analysis of Variance (ANOVA). It is the variance

between groups divided by the variance within groups. The f indicates if the experimental

treatment (i.e., companies who are tracking leading indicators in the current study) is

significantly different from the control group (i.e., companies who are not tracking

leading indicators in the current study). The f ration is derived from the ANOVA

equations (See ANOVA Table below).

ANOVA Analysis of variance is statistical technique used to measure the difference in

three or more averages.

11

8. Appendix A: Additional Analysis Tables

Tests of Between-Subjects Effects

Table 1: Model Summary for ANOVA, Dependent Variable: DART,

Independent variable: Who receives communication on leading indicators

Source Type III Sum

of Squares

Df Mean

Square

F Sig.

Corrected

Model 3.570

a 5 .714 2.743 .046

Intercept 21.779 1 21.779 83.689 .000

total_prog 3.570 5 .714 2.743 .046

Error 5.465 21 .260

Total 37.926 27

Corrected

Total 9.034 26

a. R Squared = .395 (Adjusted R Squared = .251)

Table 2: Average difference of

DART rate for companies

depending on if leading indicators

are communicated to upper

management, all management, or

all employees.

Number of

Programs

Average

DART

Upper

management

only 1.37

All management 1.44

All employees .604

12

Table 3: Average difference of DART and

OSHA rates for companies who are tracking

behavioral safety observations vs. not tracking

behavioral safety observations.

OSHA DART *

Safety_Observations

Safety_Observations OSHA DART

Not tracking Mean 2.310 1.136

N 5 5

Std.

Deviation .3799 .6602

Tracking Mean 1.784 .9589

N 19 18

Std.

Deviation 1.027 .4116

Table 4: Average difference of DART and

OSHA rates for companies who are tracking

near misses vs. not tracking near misses.

OSHA DART * Near Misses

Near_Misses OSHA DART

Not tracking Mean 2.5098 .998

N 10 6

Std. Deviation 1.93774 .103

Tracking Mean 1.037

N 22 18

Std. Deviation 1.21761 .377

13

Table 5: Average difference of DART and

OSHA rates for companies who are tracking

Training vs. not tracking Training.

OSHA DART * Training

OSHA DART

Not Tracking Mean 2.6031 1.5277

N 13 13

Std. Deviation 1.8643 1.41948

Tracking Mean 1.8643 0.9988

N 20 17

Std. Deviation 1.2486 0.79895

14

9. Appendix B: Survey Items with Summary of Survey Data Item 1:

Name of Parent Company:

Consumers Energy

Florida Power and Light

Iberdrola USA

National Grid

Integrys Energy Group

PPL Corporation

Pepco Holdings, INC

Bonneville Power Administration

Ni Source

American Transmission Company

CenterPoint Energy

Duke Energy

Southern California Edison

Northwestern Energy

DTE Energy

Henkels & McCoy

Note: Only the companies who gave permission to report their name appear in the above

list. Seventeen participants requested that their company name remain confidential and

therefore their contribution to the study is only reflected in the aggregate data below.

15

Item2:

Item 3:

If completing for a subsidiary of your company, please indicate the name:

Consumer’s Energy

Rochester Gas & Electric

New York State Electric & Gas

New Hampshire Gas

Maine Natural Gas

Central Maine Power

Wisconsin Public Service Corp

LGE-KU, PPL Montana

Northern Indiana Public Service Company

(NIPSCO)

Note: Subsidiaries are only listed for companies who gave permission to release the name

of their parent company.

Item 4:

Industry Type (select all that apply)

16

Item 5:

Company type (check all that apply)

Item 6:

17

Item 7:

Item 8:

Personal Information for contact in case clarification on survey responses is needed.

Name:

Position:

Please indicate the best way to contact you:

18

Item 9:

OSHA rate for 2012 – Recordable incidents include all work related deaths, illnesses, and

injuries which results in a loss of consciousness, restrictions of work or motion,

permanent transfer to another job within the company, or the require some type of

medical treatment.

Median OSHA rate: 1.44

Item 10:

DART rate for 2012 -- The total number of OSHA recordable cases which requires days

away from work for the year and the total number of OSHA recordable cases which

included days that employees were on transferred duties or on restricted work duty due to

an injury or illness.

Median DART: .95

Item 11:

Lost Work Cases for 2012 – The sum of deaths and Days Away from Work Cases

described above calculated from the total of the sums from OSHA for Column G and

Column H.

Median Lost Work Cases: .57

Item 12:

Severity Rate -- the number of lost days experienced as compared to the number of

incidents experienced.

Median Severity rate: 16.34

Item 13:

19

Item 14:

ds

Item 15:

20

Item 16:

Item 17:

Please indicate the safety leading indicators your company is using, if it is part of your

safety metrics, and finally if the metric is quality or quantity.

Do you collect data on the leading indicator?

Answer Options Yes No Response

Count

Safety Observations 27 3 30

Behavior Based Safety Observations 17 11 28

Safety Training 25 4 29

Hazard Resolutions 18 8 26

Corrective Action Plans 19 9 28

Safety Awards and Recognition 22 6 28

Safety Team/Committee Effectiveness 10 18 28

Tailboards/Pre Job Briefing 18 10 28

Near Misses/Close Calls 23 5 28

Preventative Health & Wellness Programs 14 13 27

Work Readiness/Pre-Shift Stretching 9 18 27

Area Audits/Safety Walkthroughs 21 8 29

Leadership Engagement 13 15 28

Safety Culture Grassroots Teams 4 22 26

Safety Culture Perception Survey 15 13 28

Employee Participation in Safety

Activities

14 13 27

Safety Work Orders 13 14 27

21

If yes, is it part of your safety metrics?

Answer Options Yes No NA Response

Count

Safety Observations 21 8 0 29

Behavior Based Safety Observations 10 12 4 26

Safety Training 13 12 2 27

Hazard Resolutions 6 15 4 25

Corrective Action Plans 7 15 3 25

Safety Awards and Recognition 5 18 3 26

Safety Team/Committee Effectiveness 6 12 5 23

Tailboards/Pre Job Briefing 7 15 3 25

Near Misses/Close Calls 11 13 1 25

Preventative Health & Wellness Programs 2 19 3 24

Work Readiness/Pre-Shift Stretching 1 18 3 22

Area Audits/Safety Walkthroughs 12 13 2 27

Leadership Engagement 7 12 4 23

Safety Culture Grassroots Teams 1 16 4 21

Safety Culture Perception Survey 9 13 2 24

Employee Participation in Safety Activities 7 12 4 23

Safety Work Orders 2 16 4 22

Do you measure quantity or quality?

Answer Options Quality Quantity Both NA Response

Count

Safety Observations 0 13 13 3 29

Behavior Based Safety Observations 2 7 8 9 26

Safety Training 1 7 15 4 27

Hazard Resolutions 0 4 8 13 25

Corrective Action Plans 0 5 8 11 24

Safety Awards and Recognition 0 7 5 12 24

Safety Team/Committee Effectiveness 1 4 3 15 23

Tailboards/Pre Job Briefing 0 5 10 10 25

Near Misses/Close Calls 3 9 9 5 26

Preventative Health & Wellness Programs 0 4 6 12 22

Work Readiness/Pre-Shift Stretching 0 3 2 18 23

Area Audits/Safety Walkthroughs 1 5 9 13 28

Leadership Engagement 0 7 4 12 23

Safety Culture Grassroots Teams 0 0 1 20 21

Safety Culture Perception Survey 2 0 10 12 24

Employee Participation in Safety

Activities 1 6 3 13 23

Safety Work Orders 0 8 1 14 23

Item 18:

Please indicate the target or goal for each leading indicator you are measuring:

Contact EEI if you would like further information on the qualitative responses to this

item.

22

Item 19:

Item 20: Please indicate the definitions for the leading indicators your organization is measuring.

Contact EEI for more information on the qualitative responses to this item.

Item 21: Please select the top three leading indicators that have been the most effective in injury prevention and rank

them in order of effectiveness from one to three with one being equivalent to the highest degree of effectiveness.

Only select your top three and mark NA for the remaining indicators on the list below.

Ranking

Answer Options 1 2 3 NA Response

Count

Safety Observations 12 1 5 12 30

Behavior Based Safety Observations 1 4 0 25 30

Safety Training 3 2 3 22 30

Hazard Resolutions 2 0 2 26 30

Corrective Action Plans 3 3 4 20 30

Safety Awards and Recognition 0 1 2 27 30

Safety Team/Committee Effectiveness 1 1 0 28 30

Tailboards/Pre Job Briefing 5 4 5 16 30

Near Misses/Close Calls 3 5 5 17 30

Preventative Health & Wellness Programs 0 2 2 26 30

Work Readiness/Pre-Shift Stretching 1 2 0 27 30

Area Audits/Safety Walkthroughs 2 2 2 24 30

Leadership Engagement 3 4 0 23 30

Safety Culture Grassroots Teams 1 2 0 27 30

Safety Culture Perception Survey 0 3 0 27 30

Employee Participation in Safety Activities 2 0 2 26 30

Safety Work Orders 0 2 0 28 30

Other 0 0 1 29 30

23

Item 22:

Item 23:

24

Item 24:

Item 25: Please provide any additional comments if you would like to share something

we did not ask.

This paper was researched, developed and written on behalf of Edison Electric Institute

and the members of their Occupational Safety and Health committee. The author’s

opinions and recommendations are based on facts gathered through the survey and their

many years of experience.

Contact EEI for more information on the qualitative responses to this question.