Assessing Safety Climate and Culture Examples Aviation ... · Assessing Safety Climate and Culture...
-
Upload
truongtram -
Category
Documents
-
view
217 -
download
2
Transcript of Assessing Safety Climate and Culture Examples Aviation ... · Assessing Safety Climate and Culture...
2012 Fifth International (HRO) Conference
Anthony [email protected]
May 2012
Assessing Safety Climate and CultureExamples Aviation, Aerospace, and Healthcare
2012 Fifth International (HRO) Conference
AGENDA
• Classic Organizational Accidents• Sources of Human Error• High - Reliability Organizations• Organizational and Safety Culture• Assessing Safety “culture” – Climate• HRO – Culture status of Selected Aviation,
Aerospace and US Hospitals• Validation of HRO – Survey Culture Instrument
2012 Fifth International (HRO) Conference
Aviation and Aerospace Accidents
• Air Florida Washington DC “start up airline”
• ValuJet Florida Everglades “outsourced cargo services”
• Challenger Space Shuttle “risk perception -- decision”
• Columbia Space Shuttle “risk perception--decision”
• TIROS weather satellite “procedural compliance”
2012 Fifth International (HRO) Conference
Key Organizational Accident Theories
• Incubation Periods (Turner,1978)• The Normal Accident (Perrow, 1984)• The High-Reliability Organization (Roberts, 1993)• Organizational Culture (Shein, 1990)• Safety Culture (Reason, 1990)• Normalization of Deviance (Vaughn, 1997)• Organizational Sense-making (Weick, 1999,
Weick and Sutcliffe 2007)
2012 Fifth International (HRO) Conference
Sources of Human Errors
• Attitudes about job safety and performance• Culture of Safety is deficient in key areas, such as
procedure compliance and reporting• Inadequate standards, training and monitoring of
practitioner qualifications • Poor communications and information flow• Non-compliance with rules, best practices• Stress, production pressure, inadequate resources
2012 Fifth International (HRO) Conference
• Failure to instill strong safety values/culture• Pressure to complete a job or meet schedule• Failure to establish or enforce standards• Over-tasking front line personnel- understaffing• Failure to manage fatigue or stress• Inadequate resources to perform job safely• Poor equipment status/reliability• Failure to manage known risks, including high-risk workers
Supervisory Contributions
2012 Fifth International (HRO) Conference
Angled DecksAviation Safety Center
Naval Aviation Maintenance ProgramRAG (FRS) Concept Initiated
NATOPS ProgramSquadron Safety Program
System Safety Aircraft DesignCRM (crew-resource-mgmt)
Aircrew reviewsRisk management (ORM)Safety climate-culture
776 aircraftdestroyed in1954
Naval Aviation Major (Class A) Flight Mishaps
Fiscal Year
7 aircraftdestroyed
inFY-11
all in flightmishaps
2000-2011
2012 Fifth International (HRO) Conference
Naval Aviation Risk Mitigation
• Recruitment and selection• Aeromedical screening• Training standardization and qualifications• Command supervision and risk management• Aircrew performance reviews• Human Factors reviews (Boards & Councils)• Crew Resource Management (CRM)• Safety Climate and Culture Assessments
2012 Fifth International (HRO) Conference
Safety Culture: The simple-minded View
Human Factors, Inc.
2012 Fifth International (HRO) Conference
High - Reliability Organization (HRO)
• A culture of trust, shared values, and risk mitigating communication processes.
• Communication that provides opportunities for open discussion and improvement.
• Distributed decision-making – Risk decisions made by most qualified
• Individual responsibility, “where the buck for safety stops everywhere.”
(Roberts, 1993)
2012 Fifth International (HRO) Conference
Emergence of Just Culture Culture Concept
The phrase just culture refers to the principles for achieving a culture in which frontline personnel feel comfortable disclosing errors—including their own—while maintaining professional accountability... A just culture recognizes that individual practitioners should not be held accountable for system failings over which they have no control.
A just culture also recognizes anyone can produce errors as a consequence of predictable interactions between human operators and the systems in which they work. However, in contrast to a culture that turns to blame as its governing principle, a just culture does not tolerate conscious disregard of rules, reckless behavior or gross misconduct (AHRQ 2008)
2012 Fifth International (HRO) Conference
Organizational Culture
Shared Values (What is important) and Beliefs(How things work) that interact with an organizations structures and control systems to produce Behavioral Norms (The way things work around here).
(Reason 1998)
2012 Fifth International (HRO) Conference
Safety Culture
• Shared values about what is safe and unsafe• Common beliefs about how to conduct safe
operations• Behavioral norms that govern risk-taking,
everyday procedures and precautions• Transmission of values, beliefs and accepted
practices to others.
2012 Fifth International (HRO) Conference
A Concise Definition of Safety Culture
2012 Fifth International (HRO) Conference
High Reliability Organization (HRO) Model1. Safety Process auditing (SPA) – conducts adequate audits and reviews of safety
processes to ensure they are working as intended.
2. Safety Culture & Reward system (SCRS) – creates a “just culture” policy and procedures for open reporting and rewards safe behavior.
3. Risk management (RSK) – has risk management processes in place and continuous risk assessment by employees at all levels.
4. Quality Assurance (QA) best practices – promotes and monitors use of standard procedures and best practices
5. Leadership and Supervision (LDSHP)-- Command Control – clearly communicates safety policies, objectives, and provides active leadership and resources to promote safe operations.
(Adapted from Libuser 1994)
2012 Fifth International (HRO) Conference
Survey is based on HRO PrinciplesMEASUREMENT AREAS: HRO MODEL
1. Safety Process auditing (SPA)
2. Safety Culture & Reward system (SCRS)
3. Quality assurance (QA)- best work practices
4. Risk management (RSK)
5. Leadership and Supervision (LDSHP)-- Command Control
SAMPLE SURVEY ITEMS
• My [organization] conducts adequate safety reviews and updates.
• Anyone who intentionally violates a safety rule is swiftly corrected.
• My [organization] has a reputation for high quality performance.
• I believe that our leaders have a clear picture of operational risks.
• My [organization] is genuinely concerned about safety.
HRO model is supported by validation tests.
2012 Fifth International (HRO) Conference
Survey based on HRO PrinciplesRoberts and Libuser HRO Model
(Adapted by Ciavarelli, 1997-2007)
1. Safety Process auditing
2. Safety Culture & Reward system
3. Quality assurance- best work practices
4. Risk management
5. Leadership and Supervision-- Command Control
Weick and Sutcliffe HRO Model (2007)
Sensitivity to operations- Awareness of operations – Mindfulness
Preoccupation with failure Leadership Commitment - promotion of a “just culture”
Reluctance to simplify – practical work processes and valid procedures
Deference to expertise – most qualified make risk decisions – management listens to workers as part of risk mitigation
Resilience – trained to detect and respond to failure and error recovery
2012 Fifth International (HRO) Conference
High Reliability Organizational (HRO) Model
SPA SCRS RSKM QA LDSHP
2012 Fifth International (HRO) Conference
Copyright © HFA
SPA QA SCRS RSK MNGT LDSHP
Copyright 2001-2007 Human Factors Associates, Inc.
Diagnostic Feedback: Fictitious Data
2012 Fifth International (HRO) Conference
Normative Data: Aviation or Aerospace
3.05%6.60%
14.23%
42.56%
33.47%
9.65% Problematic
76.03% Favorable
Survey Return Rate: 50%
2012 Fifth International (HRO) ConferencePREDICTIVE VALIDITY: Navy Safety Climate SurveyRelationship between Safety Climate and Accidents
• A study conducted by the US Navy in 2006, found that the survey item average for Risk Management, a category of the High Reliability Organization (HRO) and survey-climate scale for CSA, was a good indicator of accident risk.
• Those squadrons that took the survey were divided into four quartiles based upon their average ratings for the survey’s Risk Management items. The figure on the right shows that squadrons with the lowest HRO-climate ratings had significantly more accidents. Lowest
ClimateRatings
Highest ClimateRatings
MiddleClimateRatings
Schimpf and Figlock 2006
2012 Fifth International (HRO) ConferenceExample of Demographic Statistical Analysis
Higher Ranking Personnel Give Higher Ratings
Rank differences statistically significant (p<.001)
Over-committedShort
Resources
Survey Items
Military RankSenior Commissioned
Junior Enlisted
2012 Fifth International (HRO) Conference
BUT -- What about HRO and Healthcare?
2012 Fifth International (HRO) ConferenceProblematic %: Hospital Aviator Study Data Taken 1989 – 2001
Gaba, Singer, Sinaiko, Bowen and Ciavarelli (2003)
(Gaba, Singer et al 2003)
Survey Item
14. Senior management supports a climate that promotes patient safety.20. Senior management has a clear picture of the risks associated with patient care.22. My unit take the time to identify and assess risks to ensure patient safety.11. Patient safety decisions are made by the most qualified people, regardless of rank or hierarchy,17. I have received sufficient training to enable me to address patient safety problems.8. My performance is evaluated against defined safety standards.
OVERALL COMPARISON
NavalAviation
2.71.9
1.6
4.0
2.2
5.1
5.6
Hospitals
16.121.9
10.1
18.5
11.3
16.0
17.5
2012 Fifth International (HRO) Conference
Survey Item
3. Senior management supports a climate that promotes patient safety.
4. Senior management has a clear picture of the risks associated with patient care.
5. My unit take the time to identify and assess risks to ensure patient safety.
24. Patient safety decisions are made by the most qualified people, regardless of rank or hierarchy,
30. I have received sufficient training to enable me to address patient safety problems.
31. My performance is evaluated against defined safety standards.
OVERALL COMPARISON:
Naval Aviation
2.6
2.2
1.7
3.2
1.4
3.1
4.1
Hospitals
9.6
14.2
6.7
22.1
6.9
10.8
12.1
Problematic %:Hospital Aviator Study Data Taken 2006-2007(Singer, Rosen, Zhao, Ciavarelli, and Gaba 2010)
2012 Fifth International (HRO) ConferenceNormative Benchmark Comparison Across Domains
Comparison % Problematic MeanMajor Airlines [Data restricted by Airline Clients]Commuter Airlines [Future data needs]Freight Transport [Future data needs]Business Jet [Future data needs]Financial Services [Future data needs]
US Hospitals 12.1 unavailableUS Navy 4.10 unavailableOil and Gas helicopter transport 18.5% 3.5Air medical One 8.9% 4.0Air medical Two 7.5% 4.0Energy/Power 10.6% 3.8Aerospace 6.4% 4.1
2012 Fifth International (HRO) Conference
• Survey process for use as a risk assessment tool – validation of survey instrument is continuous.
• Approach incorporates a complete measurement, analysis and diagnostic display “dashboard” system for supervisors that provides diagnostic feedback and normative “benchmark” comparisons for a given company, industry or domain.
• Survey process was successfully used by military and civilian organizations, including helicopter transport, aerospace, airline, electric power, and critical medical care facilities. Selected survey items were used in hospital studies.
• Approach applies to other high-risk industries, that desire to asses their
• HRO performance effectiveness and risk exposure.
Demonstration Web Site: https://www.hfa-clients.com/demosite/login.html
Summary and Conclusions
2012 Fifth International (HRO) Conference
US Navy Safety Climate SurveyStatistical Validation
OPTIONAL SLIDES
• Measurement Reliability Test
• Construct Validity
• Predictive Validity
2012 Fifth International (HRO) Conference
Reliability Tests for US Navy Command Safety Assessment (CSA)
• Cronbach-Alpha -- .98
• Guttman Split-half -- .95
CSA survey is highly reliable.
Sengupta 2000
2012 Fifth International (HRO) ConferenceConstruct Validity: Factor Analysis of US Navy Command Safety Assessment Survey
Purpose: Establish factor structure of survey instrument -- key components:
• Factor 1: Safety Climate (46% of variance)• Factor 2: Hi-risk aviator management (4%)• Factor 3: Safety program effectiveness (4%)• Factor 4: Resource adequacy (3%)• Factor 5: Risk management effectiveness (3%)
About 60% of variance accounted for by less than 50 items.
Ciavarelli 2005, Sengupta 2000
2012 Fifth International (HRO) Conference
MCAS verses US Navy Accident Frequency
Safety climate Metric comparing MCAS results to individual unit mishap occurrence within 24 months after survey completion
• 168 squadrons
• August 2000-April 2004
31
Lowest quartile had nearly twice number of accidents as top quartile
(Schimpf and Figlock 2006)
2012 Fifth International (HRO) Conference
Summary of Relationship• Units with survey average scores one sigma less than average, the
risk of mishap increases 28 %
• Units with survey average scores two sigma less than average, the risk of mishap increases 65 %
• Units with survey average scores three sigma less than average, the risk of mishap increases over a factor of 2
• Units with survey average scores one sigma above average, the risk of mishap decreases 22 %
• Units with survey average scores two sigma's above average, the risk of mishap decreases 39 %
Kinzer 2004
2012 Fifth International (HRO) Conference
Civil Aviation Survey Validation Testing
• Civilian Aviation Organizational Safety Effectiveness Survey (OSES) -- derived from US Navy CSA
• Data Selected from large civilian aviation database
• Over 3000 survey takers included in a Factor Analysis Study of OSES completed in 2009.
• Results supported a five factor solution -- consistent with HRO Model used in OSES (see slide 15).
2012 Fifth International (HRO) ConferenceReferences and Source Documents
2012 Fifth International (HRO) Conference
REFERENCES AND SOURCE DOCUMENTS
Deal, T.E., Kennedy, A.A. (1982). Corporate cultures: The rites and rituals of corporate life. Reading, MA: Addison-Wesley Publishing Company.Flin, R., Mearns, K., O'Connor, P., and Bryden, R., (2000). Measuring safety climate: Identifying the common features. Safety Science, 34, 177-192.Gaba, D., Singer, S., Sinaiko, A.D., Bowen, J.D., and Ciavarelli, A.P. (2003). Differences in safety climate between hospital personnel and naval aviators. Human Factors and Ergonomics, 45,(2), 173-185.Gaba, D.M. (2000). Structural and organizational issues in patient safety: A comparison of health care to other high-hazard industries. California Management Review, 43, 83-102.Helmreich, R, and Merritt, A. (1998). Culture at work in aviation and medicine. Aldershot, UK: Ashgate Publishing, Ltd.Hofstede, G. (1994). Cultures and Organizations: Software of the mind. NY: Harper-Collins.Hofstede, G., Neuijen, B., Ohayv, D.D., Sanders, G. (1990). Measuring organizational cultures: A qualitative and quantitative study across 20 cases. Administrative Science Quarterly, 35, 286-317.
2012 Fifth International (HRO) ConferenceReferences and Source Documents
Lauber, J.K. (1993, April) A safety culture perspective. Proceedings of the Flight Safety Foundation 38th Annual Corporate Aviation Safety Seminar, pp. 11-17.Libuser, C.B. (1994). Organizational structure and risk mitigation (Ph.D. Dissertation). Los Angeles, CA: University of California at Los Angeles (UCLA).Mearns, J., and Flin, R. (1999). Assessing the state of organizational safety -- Culture or climate? Current Psychology: Research Reviews, 18, (1), 5-17.Merritt, A. & Helmreich, R.L. (1996). Human factors on the flight deck. The influences of national culture. Journal of Cross-Cultural Psychology, 27, (1), 5-24.Perrow, C. (1984). Normal accidents: Living with high-risk technologies. NY: Basic Books.Reason, J. (1998). Achieving a safe culture: Theory and practice. Work and Stress, 12, (3), 293-306.Reason, J. (1997). Managing the risks of organizational accidents. Brookfield: AshgateRoberts, K. H. (1993). Culture characteristics of reliability enhancing organizations. Journal of Managerial Issues, 5, 165-181.Roberts, K.H., Rousseau, D, La Porte, T., (1994). The culture of high reliability: Quantitative and qualitative assessment aboard nuclear powered aircraft carriers. Journal of High Technology Management Research, 5, 141-161.
2012 Fifth International (HRO) Conference
REFERENCES AND SOURCE DOCUMENTS
Sagan, S. (1993). The Limits of Safety. Princeton: Princeton University Press.Schein, E.H. (1999). The corporate culture survival guide. San Francisco: Jossey-BassSengupta, (2000, March). Factor Analysis of US Navy Command Safety Assessment Survey. Monterey, CA. Naval Postgraduate School.Turner, B.A., (1991). The development of a safety culture. Chemistry and Industry. pp. 241-243.Turner, B.A. (1978). Man-made disasters. NY: Crane-Russak & Co.Vaughn, D. (1996). The Challenger Launch Decision: Risky Technology, Culture and Deviance at NASA. Chicago: University of Chicago Press.Weick, K.E. (1987). Organizational culture as a source of reliability.. California Management Review, 19, 112-127Weick, K.E. (1993). The Collapse of sense making in organizations: The Mann Gulch disaster. Administrative Science Quarterly, 38, 628-652.Weick, K.E. and Sutcliffe, K.M.(2007). Managing the unexpected. John Wiley & Sons.
2012 Fifth International (HRO) ConferenceReferences and Source Documents
Westrum R. and Adamski, A.J., (1999). Organizational factors associated with safety and mission success in aviation environments. In, D.J. Garland, J.A. Wise, and V. David. Hopkin (eds.), Handbook of Human Factors. NJ: Lawrence Erlbaum.Weigmann, D.A., Zhang.H, von Thaden, T., Sharma,G. and Mitchell.A. ((2002,June). A synthesis of safety culture and safety climate research (Technical Report: ARL-02-3/FAA-02-2). Federal Aviation Administration, Atlantic City.Zohar, D. (1980). Safety climate in industrial organizations: Theoretical and applied implications. Journal of Applied Psychology, 65, (1), 96-102.Zohar, D. (2000). A group-level model of safety climate: Testing the effect of group climate on micro-accidents in manufacturing jobs. Journal of Applied Psychology, 85, 587-596.Zohar, D. (2001, July). Safety climate: Conceptual and measurement issues. In, Quick, J.C. & Tetrick, L.E. (Eds.) Handbook of occupational health psychology. Washington, DC: American Psychological Association .