Calhoun: The NPS Institutional Archive

Faculty and Researcher Publications Faculty and Researcher Publications

1997

Safety Climate and Risk Culture: How

Does Your Organization Measure Up?

Ciavarelli, Anthony P.

Monterey, California. Naval Postgraduate School

http://hdl.handle.net/10945/35044

Safety Climate and Risk Culture: How Does Your Organization Measure Up? 1

Anthony P. Ciavarelli, Ed.D.aciavarelli@msn.com

http://www.hfa-oses.com

1 Copyright © 2007 Anthony Ciavarelli, Human Factors Associates, Inc. All rights reserved.

INTRODUCTION

Classic Organizational Accidents

ValuJet Flight 592, referred to by its informal callsign, “Critter” never made it back to the Miamiairport. It crashed into the Florida Everglades,killing the flight crew and all passengers on board.

From this:

To this:

Accident debris: ValuJet

The National Transportation Safety Board (NTSB)found that a fire in the cargo hold of the aircraft,started when several uncapped chemical oxygengenerators ignited then burned into flight controlequipment and caused the pilot to lose control of theaircraft. The chain of errors leading to the ValuJetAirlines crash in May 1996 is one of severalexamples of an Organizational Accident or anaccident that is deeply rooted in a company’s lackof leadership commitment and support for safety.

NTSB investigators concluded that the airline, itsmaintenance contractor and the federal regulatorsall bore some responsibility for the crash. Themaintenance contractor was faulted for impropersafety procedures. The airline was held responsiblefor a lack of management oversight. The airlineargued that they were not responsible for thisaccident because it was a mistake made by themaintenance contractor personnel. But, the NTSBtold the ValuJet executives plainly, “You maycontract the maintenance work, but certainly not thesafety responsibility [1].”

Palm-90 Air Florida Crash

Perhaps the first major airline accident attributed toorganizational factors was the Air Florida, Palm-90crash into the 14th St. Bridge in Washington, D.C.that occurred on January 13, 1982.

The accident occurred following a severe wintersnow and ice storm. The airport was closed formost of the day due to the severe weather, but wasre-opened just prior to the accident. The aircraft hadnot been deiced for over two hours, yet the flightcrew prepared for immediate flight and began their

ValuJet Crash

Pilot: “What was that?”

Co-pilot: I don’t know”

Pilot: “We got some electrical problem.”

Co-pilot “Yeah. That battery charger’skicking in...

Pilot: “We’re losing everything. We needto go back to Miami.” [1]

2

take off sequence without further deicing – leavinga layer of snow and ice on the wings of the aircraft.Perhaps unknown to the flight crew, this aircraft(a Boeing 737), had several published air safetybulletins warning pilots about difficulties on takeoff with snow and ice contaminated wings. Underthese conditions the 737 had a notable tendency topitch up abruptly and stall because the aircraft couldnot achieve sufficient lift during take-off with dirtywing surfaces.

Palm-90 crashed into the Potomac,Washington DC, January 13, 1981

Co-pilot: God, look at that thing! Those don’t seemright, does it? Uh, that’s not right. (The copilot isreferring to the excessively high cockpit instrument readingsresulting from the flight crew’s failure to properly set acritical control switch during the preflight set up.)

Pilot: Yes it is. There’s eighty (knots).

Co-pilot: Nah, I don’t think that is right. … Ah,maybe it is.

Pilot: Hundred-twentyCo-pilot: I don’t know.

Pilot: vee-1, easy, vee-2 -- Forward, forward, easy.We only want five hundred. Come forward, justbarely climb (The airplane is now pitching upviolently due to loss of lift).

Co-pilot: Stalling, we’re falling!

Larry, we’re going down, Larry....

Pilot: I know it! [2]

With ice on the wings, the aircraft could notestablish sufficient take of power.

Another key finding of the accident investigationwas that the flight crew forgot to set an enginesensor switch to the “heat on”, position -- a requiredstep to be completed prior to take off, given theprevailing cold weather. Setting this switch to the“on” position was essential to prevent sensors onthe engine from freezing over due to the excessivelycold weather. As a result of this particular aircrewmistake, cockpit instruments gave erroneously highreadings. The investigators believed that the pilotreduced power to the engines because he hadmisinterpreted the erroneous instrument readings.

The Palm-90 accident was attributed to a poordecision chain as a consequence a failure of AirFlorida to give the flight crew adequate training,pilot judgment error, inappropriate procedures, anda break down in flight crew communication.

Palm-90 was one of several accidents that ledcommercial airlines to initiate training in “crewresource management” in order to instill inprofessional pilots a positive attitude and skillsrequired to improve flight crew communication.But the underlying causes of this accident wereclearly organizational in nature.

Air Florida, an upstart airline formed after the initialderegulation of the airlines in the 1980’s, had beenwoefully unprepared to handle interstate Jet flightoperations because they lacked the necessaryinfrastructure to adequately train their flight crews,and did not properly supervise flight operations.

If the term safety culture had been in vogue at thetime of the Palm-90 accident it certainly would havebeen used to describe the abominable conditionsthat lead two experienced pilots to take off with

3

snow and ice adhering to their wings, and to havelacked the necessary knowledge and skill needed tomake proper risk decisions and to follow the correctprocedures for a takeoff in such adverse conditions.

Origins of the term Safety Culture

The term safety culture actually was first applied inand accident investigation report published by theInternational Atomic Energy Agency’s NuclearAdvisory Group (AEA) -- following the disastrousmeltdown and steam explosion of the RussianChernobyl nuclear plant. The AEA concluded thatthe plant nuclear reactor itself was poorly designed,and the people operating the plant were not properlytrained or supervised. Twenty-eight people diedwithin a few months of the explosion and manymore succumbed to the residual effects of radiationpoisoning or a variety of radiation induced cancers[3].

Chernobyl

The April 1986 disaster at the Chernobyl nuclearpower plant in the Ukraine was the product of aflawed Soviet reactor design, coupled with seriousmistakes made by the plant operators in the contextof a system where training was minimal. It was adirect consequence of Cold War isolation and theresulting lack of any safety culture (World NuclearAssociation: (http://www.world-nuclear.org/info/chernobyl/inf07.htm .

The former chairman of the NTSB, Dr. JohnLauber, was perhaps the first to bring the term“culture” to the aviation community. He must haverecognized the influence of such organizationalfactors as safety culture and the supervisory

responsibility as the root cause of some aviationaccidents when he stated that, “a probable cause ofthe in flight structural breakup of the ContinentalExpress, which occurred over Texas, was the failureof this airline’s senior management to establish acorporate culture that encouraged and enforcedadherence to approved maintenance and qualityassurance procedures [4]”.

Finally, the tragic Columbia Space Shuttle accidentbrought the term safety culture and the closelyrelated concept of the high-reliability organization(HRO) to the attention of safety specialists and tothe general public’s awareness.

As most of us know, the Columbia break up uponreentry in February 2003 was caused when the heatshield protecting the spacecraft was damaged duringa launch after being struck by falling foam debrisfrom the spacecraft’s rocket fuel storage tank.

The Columbia Accident Investigation Board(CAIB) concluded that NASA’s organizationalculture and structure had as much to do with thisaccident as the falling foam. The Board determinedthat ideas coming out high-reliability theory wouldbe extremely useful in describing the culture thatshould exist in the human space flight organization.CAIB discusses differences between the U.S. Navyand NASA in terms of safety culture and operationas an HRO, and the investigators concluded that

Columbia

From Launch --

To Disaster

4

NASA could substantially benefit by following theNavy’s example of best practices [5]. So, it seemsthat the term safety culture has developed a wideaudience and is now in the vernacular of the safetyprofessionals, as well the people that work andmanage operations of aviation, nuclear energy,aerospace, and other high-hazard industries. Safetyculture then might be considered a term that can beused to depict accidents of the sort in whichleadership has failed to recognize inherentoperational risks, and has not taken appropriatemeasures to ensure the safety of key operations.

But the most egregious aspect of the organizationalaccident is the failure of management to recognizethe signs of an impending disaster. In the case ofValuJet, for example, the airline was actually underscrutiny by the Federal Aviation Agency for suspectmaintenance practices. And the Columbia accidentwas believed to be a continuation of the poor riskculture previously reflected in the 1986 Challengeraccident. In both cases, NASA employees at theworking ranks had warned their supervisors aboutthe risk of losing a flight crew due to known systemdesign flaws, falling foam (that damaged the shuttleheat shield) in the case of the Columbia and thedeterioration of O-ring seals (causing a fuel leakexplosion) in the case of the Challenger.

Safety Culture and Aviation

The use of the term safety culture has become apopular discussion topic in the aviation community,in part as a consequence of the widely publishedreports on major aviation and aerospace accidentsthat address key organizational issues.

In fact the June 2007 AIR SAFETY WORLDmagazine included two previous articles aboutsafety culture. The first article High culture: The artof creating and sustaining a beneficial safetyculture in different setting was written by Peter V.Augur, Jr. He gave a thorough review of theconcept of safety culture and discussed the role thatsafety culture plays in aviation organizations [6].

The second article, written by, Linda Werfelman,and Anne Paylor entitled. Keys to Safety: SafetyCulture compared US and European Air TrafficControl (ATC). In this article the authors suggestthat the United States and the European ATCorganizations represent two distinct safety cultures.The author concluded that the Europeans are aheadof the USA in the promotion of various aspects of astrong safety culture, such as open (non-punitive)reporting of hazards and willingness to discussoperational errors, or close-calls such as near midaircollisions or runway excursions. [7]

Yet, in spite of the professional’s use of the termsafety culture and many popular notions about it,there is no widely accepted definition of safetyculture. Moreover, there is no common metric formeasuring the strength of a particular safety culture.Therefore there is no clear method by which anorganization can assess its safety culture ordiagnose its particular strengths and weaknesses.This lack of methodology results in some frustrationon the part of the executives who must manageorganizations that necessarily operate in hazardousenvironments to conduct their business.

An adage often made by executives is that, “if youcan’t measure it—you can’t manage it.”

US Naval Aviation Safety Culture Study

The US Navy responded to the challenge ofmeasuring and managing safety culture in 1996,when their Naval Aviation squadrons experienced arash of accidents that they attributed to “HumanFactors.” One accident, in particular, captured theNavy’s attention. A Navy jet, manned by twoexperienced naval aviators, crashed on take-offfrom an airport in Tennessee, killing the pilot andhis back seat flight observer, as well as severalcivilians on the ground. Navy investigators of thisaccident pointed to the failure of the pilot’soperational command to adequately manage a wellknown high-risk pilot.

5

F-14 Crash in Tennessee

The pilot in the Tennessee crash was previouslyinvolved in another serious aircraft accident, and hewas considered by his peers to be a marginalperformer. Just prior to the accident, he had justcompleted a cross-country flight and visit to hishometown. During his return flight take off, hisparents were on the side of the runway watchingwhile the pilot attempted a dangerous high-angletake off to impress his spectators. The pilot flewinto some low clouds and lost visual contact withthe ground below. He became spatially disoriented(where he could not tell up from down). The pilotchose the wrong direction and then crashed into aresidential complex.

Following the Tennessee accident, the author servedas a member of a blue-ribbon panel from 1996-2004formed by the US Navy to study the underlyingcauses of Naval Aviation aircraft accidents. Thepanel took recognition of the fact that in theTennessee accident, and other accidents like it, theinvestigators would encounter comments by somesquadron members like, “well, we could all see thiscoming – that pilot was clearly a high-risk aviator--and it was only a matter of time.”

Then upon investigation, accident boards wouldfind corroborating information about a particularmarginal pilot or some other known circumstancesunderlying an accident that produced risks whichwere not appropriately managed by the leaders inthe command. Had the organization’s leadership

understood the risk and had they responded andtaken proper measures to reduce the risks, aparticular accident might not have occurred.

This panel conducted a thorough review of NavalAviation accident causes and concluded thataccidents of this type were very similar in kind tocivilian accidents like ValuJet, Chernobyl and theSpace Shuttle. This finding led to the developmentof the initial survey instrument designed to assesssafety climate and safety culture, and other relatedorganizational factors of aviation squadrons.

US Naval Aviation Study

The author and his colleagues from the NavalPostgraduate School in Monterey California andUniversity of California at Berkeley set out toexplore innovative methods for assessing suchorganizational factors, as safety climate and safetyculture [8, 9]. The rationale behind this effort wasbased on the finding that many US Naval Aviationaccidents traced their roots to a lack of leadershipcommitment and management oversight – resultingin a poor command climate, a weak safety culture,and risky supervisory practices.

The US Naval Postgraduate School faculty workedin collaboration with Professor K.R. Roberts,University of California at Berkeley, to construct anorganizational safety effectiveness employee surveybased on Robert’s theory of a High ReliabilityOrganization or HRO. Roberts and her colleaguessaw a common thread of factors that cut acrossorganizational accidents like the Columbia,Chernobyl and ValuJet, and others. In each of thesecases there was a general lack of supervisorycontrol resulting in the failure of employees to makeappropriate risk decisions, or to follow the correctprocedures required to operate safely [10,11].

Basically, Roberts and her colleagues believe thatsome organizations are able to operate more reliablythan others because they place a higher value onsafety and greater focus on avoiding failure.

6

Roberts conducted field studies aboard US NavalAircraft Carriers, Air Traffic Control facilities, andUS nuclear plants. These kinds of organizationshave learned through their history how to bestmanage their risks. Some of the characteristics thattypify high-reliability organizations (HRO’s) are:

Accurate perception of hazards andoperational risks

Commitment and involvement of allmanagement levels in safety

Open reporting of any unsafe condition orrisk situations

Good communications up and down thecommand chain

Continuous training, with high performancestandards in place

A culture of trust between workers and theirsupervisors

US Navy Survey based on HRO Theory

Naval Postgraduate School Professors Ciavarelliand Figlock developed a Safety Climate Surveybased upon key principle drawn from HRO theoryand early work by related to measurement of safetyclimate. The initial survey developed was called theCommand Safety Assessment (CSA) Survey [10].

An Internet, Web-based, version of the CSA wasdeveloped and now is in regular use by all US Navyand Marine Corps aviation units. The online surveyenables respondents to submit voluntary andanonymous inputs regarding their individualattitudes and opinions about their organization’ssafety climate based upon criteria or characteristicsattributed to an HRO. The CSA was developedspecifically for Naval Aircrews, and its counterpartthe Maintenance Climate Assessment (MCAS) wasdeveloped later for aircraft maintenance personnel.

Each survey asks Naval Aviation personnel to ratethe command based on their perception of the unit’sperformance as an HRO.

Safety Climate and Culture

Later versions of the CSA incorporated variousaspects of safety culture and safety climate into theSurvey, based on the work of European socialscientists. These researchers, working primarily inthe Oil and Gas Industry greatly improved ourunderstanding of the differences between safetyculture and safety climate. They were careful todistinguish the meaning of the terms safety cultureand safety climate [12, 13].

Culture is considered to be the force behind anorganization’s goals, and it drives the means toattain goals, and how to achieve success. Anorganization’s cultural values also guide thedecisions and actions for correcting deviations fromnorms and expectations [14, 15].

Safety culture is defined as the shared values,beliefs, assumptions, and norms that governdecision-making and that may affect individual andgroup attitudes about danger, safety, and the properconduct of hazardous operations.

Safety climate is an important indicator of theunderlying safety culture.

Safety climate refers to the shared perception of thepeople in an organization that their leaders aregenuinely committed to safety of operations, andhave taken appropriate measures to communicatesafety principles, and to ensure adherence to safetystandards and procedures [16, 17].

Safety climate is considered to be a “snapshot” ofthe prevailing attitudes among organizationalmembers about management’s commitment tosafety, given the underlying culture. It representsthe perception of people about their organization,taken under the conditions found in the workplaceat the time that a particular survey was completed.

The CSA/MCAS survey assessments developed forthe Navy, and later applications in civilian aviation,aerospace and healthcare, are intended to measure

7

safety climate and to determine the overall safetyeffectiveness at the organizational level. Surveyitems are designed to address key aspects of safetyclimate, including people’s perceptions regardingmanagement’s commitment to safety, the efficacyof risk management processes, attitudes about thework environment and opinions about safetyprogram effectiveness. Results from the surveysometimes can be used in the estimation of accidentrisk. The risk of an “organizational accident” isbased on the extent to which the organizationexhibits HRO attributes of leadership commitmentto safety, adequacy of policies, adherence tostandards and other factors [18]. .

The application of the HRO-based safety climatesurveys in selected US Naval and Marine Corpsunits has been very well received, and serves as animportant source of performance feedback forNaval Commanders in understanding the state oftheir safety climate, for judging their safety culture,and for improving the safety of squadron operationsby providing the opportunity to intervene and breakthe chain of events that might lead to an accident.

Since its inception in 1997, the CSA alone has beenadministered to all US Navy and Marine Corpssquadrons, with over 80,000 survey respondents inthe Navy’s CSA database at this writing.

The success of the Navy’s CSA led to numerousdemands for similar survey approaches in civiliancompanies, and eventually led the author to developcomparable online survey systems that are nowused across the world in civilian aviation, aerospaceoperations, and some healthcare industries.

Examples of Safety climate Findings

Over the years survey methods for assessing safetyclimate and culture have matured to provideorganizations with reliable and valid measures.Findings from such surveys have been consistentand useful. For example, Figure 5 shows the resultsof the Navy’s CSA survey item ratings taken fromover 10,000 respondents and shows a comparison

across military ranks. The chart shows that muchhigher ratings are given by senior rankingcommissioned officers (04-06- Navy Commander-Captain) compared to lower officer ranks (01-03-Lieutenant to Lieutenant Commander). This relationshipalso held for the non-commissioned officers. Higherranking enlisted officers (E6-E9- Petty Officers) gavehigher ratings than the lower ranks (E1-E5 -Seaman)

The application of similar safety climate surveyshas shown comparable results for the civilianaviation, aerospace and healthcare industries.Climate ratings are always consistently higheramong an organization’s supervisors compared toworkers. This finding suggests that manysupervisors might not be well-connected to thethoughts and feelings of their subordinates when itcomes to the perception of safety climate and thestrength of their culture.

Validating the Measurement Approach

Professional survey developers place a high valueon the worth of surveys in terms of “statisticalgoodness.” Two of the important aspects of the“statistical goodness” of a survey are the survey’sreliability and validity.

Reliability refers to the dependability of measuresobtained in a particular survey. Basically, when asurvey is reliable this means that the results arerelatively error free and consistent – so that if thesurvey were administered twice to the same peopleat the same time, the ratings would be identical.

Establishing the reliability of measures is one ofthe first steps in determining the usefulness of asurvey. Unless the survey can be shown to bereliable, by using a variety of statistical methods,there would be no point in attempting to show thatthe survey is valid for its purpose. Reliability is anessential requirement for establishing the “validity”of the survey. If the obtained ratings were “jumpingall over the place” i.e. appearing as random andunreliable, then you simply could not trust thefindings, and could not draw valid conclusionsabout any specific results.

8

The statistical reliability of both US Navy andcivilian surveys was found to be very high,reflecting a good amount of response consistencyamong survey takers for multiple applications [18].

The term “validity” refers to a different issue aboutthe survey measures. We want to believe that we arereally measuring what we set out to measure – inthis case safety climate and its close relative safetyculture. To meet this particular validity requirementthe survey instrument itself must be carefullyconstructed to reflect the key attributes of climateand culture.

This kind of validity is called “content validity”,and focuses on inclusion of survey items that reflectsome of the underlying organizational dimensionslike climate, culture, or HRO attributes.

Here are some examples of survey items, based onadherence to content validity:

1. My command has a realistic view of or ouroperational risks.

2. The leadership in this command is verycommitted to safe operations.

3. All levels of management are actively involved inkeeping us safe.

4. I am not reluctant to report an unsafe condition ora high-risk incident.

5. Deliberate violations of rules or standards arevery rare in my unit.

A well designed survey will not only ask the rightquestions, but will prove useful in diagnosing anorganization’s strengths and shortcomings.Sometimes the goal of diagnostic analysis isachieved by organizing survey items into specificmeasurement areas that reflect different componentsof climate, culture or set of HRO attributes. For theCSA, the categories of the survey were borrowedfrom studies by University of California researchers

Karlene Roberts and Carolyn Libuser, and wereadapted for our purpose [19].

Five categories, representing different componentsof an HRO are:

1. Safety Process Auditing (SPA) -- a system of checks andreviews to monitor and improve processes2. Safety Culture and Reward System (SCRS) -- socialrecognition that reinforces desired behavior, or correctsundesired behavior3. Quality Assurance (QA) -- policies / procedures thatpromote high-quality performance and work performance4. Risk Management (RSK MNGT) – whether or not theleaders correctly perceive operational risks and take correctiveaction5. Leadership and Supervision (LDSHP) – policies,procedures, and communication processes used to improvepeople’s skills and to proactively manage work activities andoperational risk

Making good use of survey data

After the survey is administered, the results can bereviewed as an overall survey outcome (averagerating across all survey items), or the results can bebroken down and compared by each individualHRO component (safety process auditing, safetyculture-reward system, etc.). Figure 1 shows anexample of the “supervisor’s dashboard display”,which depicts the survey results by examiningsurvey item averages, and percent agreement. Thedisplay is segmented, and color coded, so that thesupervisor can see the statistical summary datadivided by the five key components of the safetyclimate survey – Safety Process Auditing (SPA),Quality Assurance (QA), Safety Culture andReward System (SCRS), Risk Management (RSKMNGT), and Leadership and Supervision (LDSHP).

Figure 2 shows the Normal Distribution Curve, or“Bell curve”, that illustrates how survey ratings canbe viewed against a specific average or centralvalue in the normal distribution. Using this diagram,and a little statistical computation, a specificorganization (company or department within acompany) can compare its results to an overallaverage or norm.

9

The norm can be based on a particular company’saverage rating or on the average for that particularindustry. With this “normative” approach it wouldbe possible to establish norms for the entire airlineindustry or another sector of the aviation industry,like Air Traffic Control. And, with enough surveydata we would be able to establish norms, orstandards for different industry sectors likeAerospace, Oil and Gas Extraction, and Healthcare.

Figure 3 – Illustrates how normative information ona Supervisor’s Display can be presented. The surveyresults feedback display shows a typical bar chartwith agreement percentages along the five - point(1-5) Likert Rating Scale. This display also shows abell curve indicating the placement of a specificorganization’s average rating on the bell curve.This placement allows a particular organization,like a single company department, to compare itsown organization’s average to an overall companyaverage or norm. Figure 4 – Percent graphs andnormative display curve for selected survey items.

Final Survey Validation Process

Once we have a reasonable sample of survey data,we can approach another aspect of “statisticalgoodness” – that of “predictive” validity.

We would expect organizations with a good safetyclimate and strong culture to have a better safetyrecord than those organizations that do not. In fact,this is exactly what we have found out in examiningthe safety climate ratings across many surveys takenover the past few years in Naval Aviation. Figure 5and Figure 6 show different aspects of therelationship between safety climate ratings andsafety performance outcomes - measured in termsof the aircraft accidents occurring in a definedperiod between survey administration and accidentoccurrences [21, 22]. Figure 6 shows the frequencyof naval aircraft accidents (A, B, and C levels ofaccident severity (running from A most serious to Cleast serious accident), for each of four equalintervals (quadrants) along the safety climate scale.

Figure 5 presents some results that show accidentsoverlaid on one specific survey component (RiskManagement Scale of the CSA).

As can be seen from both Figures 5 and 6, there arelarge differences in accident occurrences along thesafety climate scale. These Figures showdifferences that indicate more favorable safetyperformance outcomes (less accidents) for thoseunits scoring higher on safety climate ratings.

In figure 5, for example, the units in the lowestquartile (interval 2.90-3.59) of the MaintenanceClimate Assessment Survey (MCAS) had nearlytwice the number of accidents (94 vs. 49) in the 24-month time frame [20].

In figure 6, derived only from the CSA RiskManagement subscale, there is a clear relationshipbetween the ratings of potential risk exposure,judged by respondents and the accident frequency –the lower the rating on the risk management scale,the more accidents the unit had for the definedperiod [21].

Aviation and Medicine Comparison

In another application of high-reliability basedsafety climate measurement we compared ratingsobtained on the Navy’s CSA to ratings obtained ona similar survey given to a sample of hospitals [22].

Figure 7 shows an example of the results obtainedwhen 23 common Likert scale survey items used bythe Navy and in the hospital study were comparedin terms of the number of defined “problematicresponses”, i.e. survey responses that should havebeen answered favorably, but were not, indicating afair-to-poor safety climate. The overall problematicresponse rate was about 6% for naval aviationverses 18% for hospitals [22, 23]

Authors of the US Navy-Hospital study concludedthat the perception of naval personnel was far morepositive because the US Navy has had a longerhistory of focusing on potential failures and hasformulated specific processes over the past 60 years

10

or so to ensure that their command leadership isactive in the prevention of accidents.

Conclusions and Implications

Accidents across a variety of industries, includingaviation, aerospace, and healthcare appear to exhibitthe same or similar factors underlying what hasbecome know as an “organizational accident.”

The US Navy and several high-hazard civilianindustries in aviation, aerospace and healthcare nowrecognize the influence of such organizationalfactors as safety climate and culture on the theirsafety performance. The leaders of theseorganizations have sought a means to determine thestate of their safety climate and culture, asperceived by their employees – because it isunderstood that the people “on the ground” areclosest to the daily routines and risk issues that mayprevail in any high-hazard business operation.

Safety Climate-Culture Surveys are one means formanagers to keep abreast of hazards and risksinherent in their organization. Survey results canprovide the opportunity for an organization toidentify possible unknown risks and to intervene intime to prevent accidents and man-made disasters.

REFERENCES

1. NTSB Accident Report: In-Flight fire and impactwith terrain ValuJet Airlines Flight 592, DC-9-32,N904VJ, Everglades, Near Miami, Florida, May 11,1996.Washington DC: National TransportationSafety Board

2. NTSB Accident Report: Air Florida, Inc. Boeing737-222, N62AF Collision with 14th Street Bridge,Near Washington, D.C., January 13, 1982. NationalWashington, D.C. Nat. Transportation Safety Board

3. INSAG Report on the Chernobyl NuclearAccident (INSAG-3), 1989. Vienna.

4. NTSB Accident Report: Britt Airways, Inc.,Continental Express Flight 2574 in-flight structuralbreakup EMB-120RT, N33701, Eagle Lake, Texas,September 11, 1991. Washington D.C. NationalTransportation Board (p. 54)

5. NASA. Columbia Accident Investigation Board:STS-107, August 2003 (pp.177-181).

6. Snyder, G.H., Jr. (2007, June). High culture incorporate aviation. Aero Safety World – The Journalof the Flight Safety Foundation, Alexandria, VA:Flight Safety Foundation

7. Werfelman, L and Paylor, A. (2007, July). Keysto safety. Aero Safety World – The Journal of theFlight Safety Foundation, Alexandria, VA: FlightSafety Foundation

8. Ciavarelli, A.P. and Figlock, R. (1997).Organizational factors in Naval Aviation accidents.In, Proceedings of the International AviationPsychologists Conference, Columbus, Ohio.

9. Ciavarelli, A., Figlock, R., Sengupta, K., &Roberts, K. (2001). Assessing organizational safetyrisk using safety survey methods. The11thInternational symposium on AviationPsychology, Columbus, OH.

10. Roberts, K.H. (1990, summer). Managing high-reliability organizations. California ManagementReview. 32, (4), 101-113.

11. Roberts, K.H, and Bea, R. (2001). Mustaccidents happen? Lessons from high-reliabilityorganizations. Academy of Management Executive,15, 70-78.

12. Flin, R., Mearns, K., O'Connor, P., and Bryden,R. (2000). Measuring safety climate: Identifying thecommon features. Safety Science, 34, 177-192.13. Mearns, Whitaker, S.M., & Flin, R. (2003).Safety climate, safety management practice andsafety performance in offshore environments. SafetyScience, 41, 641-680.

11

14. Schein, E.H. (1990). Organizational culture.American Psychologist, 45, (2), 109-119.

15. Schein, E.H. (1996). Culture: The missingconcept in organizational studies. AdministrativeScience Quarterly, 41, 229-240.

16. Zohar, D. (1980). Safety climate in industrialorganizations: Theoretical and implications. Journalof Applied Psychology, 1), 96-102.

17. Zohar, D (2002). Safety climate: Conceptualand measurement issues. In J. Quick & L. Tetrick(Eds.), Handbook of occupational psychology (pp.123-142).Washington DC: APA.

18. Ciavarelli (2007). Assessing safety climate andorganizational risk. In, Proceedings of the 51st

Human Factors and Ergonomics Society Meeting.Baltimore, MD.

19. Libuser, C.B. (1994). Organizational structureand risk mitigation (Ph.D. Dissertation). LosAngeles, CA: UCLA

20. Schimpf, M.W., and Kinzer, C.M. (2004). AStudy of the relationship between the MaintenanceClimate Assessment Survey (MCAS) and US NavalAircraft Mishaps, Monterey, CA: School ofAviation Safety, Naval Postgraduate School.

21. Schimpf, M.E., and Figlock, R. (2006).CSA/MCAS Surveys and their relationship to NavalMishaps. Norfolk, VA: US Naval Safety Center.

22. Ciavarelli, A.P. (2005, September). AssessingSafety Climate and Culture: From Aviation toMedicine. St. Louis University.

23. Gaba, D.M., Singer, S.J., Sinaiko, A.D., Bowen,J.D., & Ciavarelli, A.P. (2003). Differences insafety climate between hospital personnel andNaval Aviators. Human Factors and Ergonomics,45, 173-185.

ABOUT THE AUTHOR

Anthony Ciavarelli, Ed.D is a Professor of AppliedPsychology at the Naval Postgraduate School,Monterey California where he teaches and performsresearch in Human Factors covering various aspectsof human performance in aviation systems. Dr.Ciavarelli has over 35 years experience, includinglead technical positions in the Aerospace – DefenseElectronics Industry and Academia. He is mostnoted for his recent work in assessing safety climateand safety culture in military and civilian aviationand aerospace systems. He founded Human FactorsAssociates, Inc., a California Corporation thatspecializes in evaluation of safety culture, riskmanagement, and organizational effectiveness.

12

Diagnostic Feedback: Fictitious Data

2 0 %

3 0 %

4 0 %

5 0 %

6 0 %

7 0 %

8 0 %

9 0 %

1 0 0 %

1 3 5 7 9 1 1 1 3 1 5 1 7 1 9 2 1 2 3 2 5 2 7 2 9 3 1 3 3 3 5 3 7 3 9 4 1 4 3 4 5 4 7 4 9 5 1 5 3 5 5 5 7 5 9

2 ,0

2 ,5

3 ,0

3 ,5

4 ,0

4 ,5

5 ,0

SPA QA SCRS RSK MNGT LDSHP

Figure 2 - Norming Survey Data

Figure 1 - Sample Results FeedbackDisplay

Rating is 1 standardunit below mean (X) orat the 14th percentile(left side - dark blue).86% of scores (rightside - light blue) areabove this value.

Supervisors can use this “dashboard” to review survey results displayed as percent favorableresponses (left side of chart), average ratings (right side of chart). The survey items are on thebottom of the chart and bars are color-coded to show different sections of the survey.

(V. Coletta, Dashboard Design)

13

Figure 3 shows the overall results of a specific organization in terms of the percentage of unfavorable tofavorable responses (left side chart). The right side of this display shows the bell curve with the placement of an

Figure 3 - Supervisor Feedback Display Showing Results Compared to a NormSupervisors can view a normal Bell curve diagram that shows how personnel ratetheir specific organization in comparison to the average for all survey ratings.

14

organization’s “Western Division” with respect to an overall average. In this hypothetical case, this organizationscores in the 45.66 percentile – just below average which would be the 50th percentile.

Figure 4 - Supervisor’s Feedback Display Showing Selected Survey Item Results

The above charts show hypothetical results, in which specific survey items are compared to an overall average.

15

Figure 5: Differences in safety climate ratings among military ranks

Average CSA Scores by Military Rank

2

2.5

3

3.5

4

4.5

5

1 4 7

10

13

16

19

22

25

28

31

34

37

40

43

46

49

52

55

58

61

O4 - O6O1 - O3E6 - E9E1 - E5

Figure 6: Safety Climate verses Accidents:Months

MCAS Survey vs Mishaps within 24 Months after Survey

21 1912 13

12 13

93

61

46

33

33

0

10

20

30

40

50

60

70

80

90

100

2.90-3.59 3.59-3.72 3.72-3.82 3.82-4.26

Quartiles based on Survey Average

Mis

hap

s

Class C

Class B

Class A

94

76

5449

(Gaba, Singer, Sinaiko, Bowen, and Ciavarelli, 2003)

Figure 8: % problematic ratings: Naval Aviation vs. Hospitals

Clim ate

Comm.

mgm t flex

safety issues

0

5

10

15

20

25

30

NavyHospitals

Bottom Mid-Low Mid-High Top

Operational Squadrons

All Squadrons

10

6

7

5

9

4

3

2

0

1

2

3

4

5

6

7

8

9

10

Cla

ss

AF

Ms

wit

hin

On

eY

ear

Quartiles based on Risk Management Average

Figure 7: CSA Risk Management Scale and Accidents(Schimpf & Figlock, 2006)

US NAVY SURVEY STUDIES:Statistical Results

Schimpf & Kinzer 2004