Measuring Safety Cultures

Keith D. Johnson, MS President, Consitrium Group Inc.

April 20th , 2010

Why Measure? Jack Welch

“There are only three measurements that tell you nearly everything you need to know about your organization’s overall performance: employee engagement, customer satisfaction and cash flow.” To Welch, the conclusion was clear: Without energized employees, no company can win over the long run.

Business Week

Survey Types• Basic types of surveys • Satisfaction or Engagement

• Survey Structure• Emotional or Rational

• Survey Focus• Compliance (OSHA-based questions)• Systems (Safety systems questions)• Organization (Measures employee engagement)

• One assessment cannot address everything!

Current trends are employee engagement assessments with emotional-based questions focused on the organization.

Assessment Details

• Yes and No questions• The “Likert Scale” (1 to 5)• Interviews (Cognitive issue)• Sample Size • Comments Section• Number of Survey Questions• Anonymity is very important

The Questions

Question development is the most overlooked and misunderstood aspect to developing a truly valid assessment. The questions MUST be:

• Single Barrel questions only• Validated• Tested (For accuracy and strength)• Actionable• Surgical in nature

Question Examples

• Emotional• My supervisor demonstrates a commitment to safety.

• Rational• I am paid fairly for the work I perform.

• Systems• My company has a near-miss reporting procedure.

• Compliance• I received quality hazardous materials safety training.

Question Examples

• Upstream Example• My supervisor communicates critical safety information to me.

•Downstream Example• I communicate critical safety information to my employees.

Avoid This!Managers, supervisors and team leaders have the training & experience to lead, train and motivate employees to perform safely and when all else fails they use enforcement actions to ensure compliance with company rules.

Dr. Stefano Monti, BROAD Institute

Question Testing

Correlations

• None• Simply look at question responses

• Direct• Management, communication, training, etc

• Multi-faceted• Complex interrelationships

Value of Sorting Options

• Sorting options include the following:• Trade or craft• Supervision• Management• Time• Age Group• Division• Region

Analysis Process

Is it a true analysis of results?

• What is an accurate sample size?• Do I add a “weighting”?• Are there correlations?• Gap analysis & gap range• Standard deviations & trending• Is there a database to support results?

You Need a Database!

Can you manage the data?• You have surveys for Executives, Management, Supervision and Craft• You have 10 sorting options (project, trade, responsibility, years, etc.) • You have a survey of 40 statements• You used the Likert scale for responses • You have 300 completed surveys

40 statements on a 5 point scale equal 200 possible responses per survey, multiply this by 300 surveys and you have 60,000 data points.

And you still have not added the survey type or sorting options!

Sample Results

Sample Results

Sample Results

Off-the-Shelf Surveys

• Benefits• Price

• Drawbacks• Valid to your industry? • Structure focus? (i.e. emotional/rational)• Tested?• Validated?• Benchmarking ability?• Flexibility?

The Process• Determine Survey Focus• Develop the survey• Test Survey• Validate Survey• Communicate Intentions• Administer Survey• Complete Analysis• Report Results• Take Action• Re-evaluation

Success Stories

• Miller Pipeline• Canada mine supervision• Roel Construction• Sundt Construction• Drinking on the job (Madison Ave & Power Plant)

• URS Construction & Energy• ADM• Shaw Group plant comparison

Summary

• Determine survey type & focus• Choose sorting options• Remember, the questions are very important• Use only single-barrel questions• Are the questions valid?• Have questions been tested?• Are the correlations relevant? • Does it fit with your industry?• Database requirements/ability• Benchmarking data

Conclusion

Questions

Keith D. Johnson, MS President, Consitrium Group Inc.keith.johnson@consitrium.com

952-212-8431