Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not...

64
Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure to Control Cri=cal Material Items Fraud Brian HughiB Office of Safety and Mission Assurance

Transcript of Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not...

Page 1: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Common Threads Among Catastrophic Mishaps

LessonsNotLearnedVulnerableDesign

WorkmanshipShortcomingsProcessControlFailures

FailuretoControlCri=calMaterialItemsFraud

BrianHughiB OfficeofSafetyandMissionAssurance

Page 2: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

PrecursorEventIneffec/veCorrec/veAc/on

VulnerableDesign

Fraud Workmanship/ProcessControlShortcomings

MaterialControlInadequacies

BigDigTunnelCollapse

X X X X

TurkishAirlinesFlight981

X X X

USSThresher

X X X X

Apollo1

X X X X

USSIwoJima

X

Orbi=ngCarbonObservatory/Glory

X X X X

AntaresOrb-3

X X

Falcon9-20

X2

Page 3: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

•  7.5milecorridor•  161lanemiles•  5milesoftunnel•  6interchanges•  200bridges•  541,000truckloadsofdirt•  3.8millioncubicyardsofconcrete

Originallyscheduledtobecompletedin1998atacostof$2.6billion, theprojectwasfinallycompletedin2006atacostof$14billion.

3

The Big Dig

Page 4: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

OnSeptember9,1999,aconstruc=oncontractoremployeeinstallingven=la=onductworkoverthetunnelceilingno=cedthatseveraloftheanchorshadbeguntopullout.

The Smoking Gun

OnNovember12,1999,aproofloadtestwasperformedononeoftheanchorsthathadshownsignificantnine-sixteenths-inchdisplacement.Theengineernotedthat“theboltheldforafewseconds,thenbegantopulloutwithalmostnoresistance”.

4

The Gi;

Page 5: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

5

Page 6: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

The Supplier’s Response

Whenthesupplierwascalledtoexaminetheanchordisplacements,theyseemedsurprisedthattheanchorsthathadbeensuccessfullyprooftestedonlyafewmonthsbeforecouldbefailing.Installa=onproblems(e.g.,excessivepreload)werepostulatedasthecause.

Noevidencewasfoundthatthesuppliertookanyfollow-upac=onadertheexamina=on.

-Nofurthertes=ng

-Nofurtherresearch

“At least some supplier officials were aware that their Fast Set epoxy was subject to creep, but this information was apparently not considered or was not known by the representatives who evaluated the failed anchors. Even if the information about poor creep resistance was not common knowledge, a reasonable amount of research would likely have revealed it. The Safety Board would have expected the supplier of a safety critical component to have been more proactive in determining why its product was failing.”

Page 7: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

The Builder, Design Agent, and Project Manager Reply

Increasedproofloadtes=ng

Therootcauseforthehangerdisplacementwasneveriden=fied……andsurveillancemonitoringinspec=onswereneverimplemented.

Page 8: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

20,000#

15,000#

10,000#

5,000#

0

LbF

Design Service Load Post Installation

Proof Test Finite Element Analysis

2,600 #

Calculated design service load

(2,600 Lb-Force)

Page 9: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

20,000#

15,000#

10,000#

5,000#

0

LbF

Design Service Load Post Installation

Proof Test

Finite Element Analysis

2,600 # 3,250 #

After each bolt was installed, a proof test was conducted at 25% higher than design service load

(3,250 Lb.- Force)

Page 10: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

20,000#

15,000#

10,000#

5,000#

0

LbF

Design Service Load Post Installation

Proof Test Finite Element Analysis

2,600 #

6,350 #

3,250 #

Later, after slippage was noted, bolts were proof tested to the maximum allowable load

(6,350 Lb-Force)

Page 11: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

15,000#

10,000#

5,000#

0

LbF

Design Service Load Post Installation

Proof Test Finite Element Analysis

2,600 # 2,371 # 2,823 #

6,350 #

3,250 #

A finite element analyses determined that the load would be between 2,371 and 2,823 lb force

Page 12: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

"You’ve noted the key piece of information that is missing. That is the

cause of the anchor failure and how the repair procedure will overcome that… We are not trying to hold up construction, we are trying to make a determination that the installation is safe…” Design Manager e-mail concerning response to Deficiency Report

“Glaringly absent from the Deficiency Report is any explanation why the anchors failed and what steps are proposed to ensure that this problem does not reoccur.” Structural Engineer e-mail reply

Page 13: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

The Gi; (Part 2)

OnDecember17,2001,aqualitycontrolinspectorsubmiBedaNoncomplianceReport,whichstated:

“Severalanchorsappeartobepullingawayfromtheconcrete.Thesubjectanchorswerepreviouslytestedtotherevisedvalueof6,350pounds,allofwhichpassed.[…]Reasonforfailureisunknown.”

“Atthispoint,itshouldhavebeenobvious[…]thattheremedythathadbeendevelopedinresponsetotheanchordisplacementinthe[HighOccupancyVehicle]tunnelin1999hadnotbeeneffec/ve,asanchorsthathadpassedprooftes/ngathighervaluesweres/lldisplacing.Thiswasanotheropportunityto[…]inspectalltheinstalledanchorstodeterminetheextentand,moreimportantly,thecauseoftheanchordisplacement.Instead,thecompaniesapparentlyconsideredthecon/nuingfailuresasisolatedinstancesandtooknoac/ontoaddresstheprobleminasystemicway.”

NTSBAccidentReport

13

Page 14: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Accident Synopsis

At11p.m.onJuly10,2006,a1991Buickpassengercaroccupiedbya46-year-oldmaledriverandhis38-year-oldwifewastravelingeastboundintheI-90connectortunnelinBoston,MA,enroutetoLoganInterna=onalAirport.Asthecarapproachedtheendoftheconnectortunnel,asec=onofthetunnel’ssuspendedconcreteceiling(26tons)detachedfromthetunnelroofandfellontothevehicle,crushingitsrightside.

14

Page 15: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

ProximateCause

Useofanepoxyanchoradhesivewithpoorcreepresistance

15

NTSB Accident Report

Page 16: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Creep

ASTMD2990-01 Sincetheproper=esofviscoelas=cmaterialsaredependenton=me…aninstantaneoustestresultcannotbeexpectedtoshowhowamaterialwillbehavewhensubjectedtostressordeforma=onforanextendedperiodof=me.

Epoxyisapolymeranditss=ffnessis=meandtemperaturedependent.Ifaloadisappliedsuddenly,theepoxyrespondslikeahardsolid.Butifthatloadisthenheldconstant,themoleculeswithinthepolymermaybegintorearrangeandslidepastoneanother,causingtheepoxytograduallydeform.Asthedeforma=onincreases,itbecomesirreversible.

Page 17: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure
Page 18: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure
Page 19: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Epoxy Secured Bolts

Page 20: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

CreepTheUnknownKnown

20

“At least some supplier officials were aware that their Fast Set epoxy was subject to creep, but this information was apparently not considered or was not known by the representatives who evaluated the failed anchors. Even if the information about poor creep resistance was not common knowledge, a reasonable amount of research would likely have revealed it. The Safety Board would have expected the supplier of a safety critical component to have been more proactive in determining why its product was failing.”

NTSB Accident Report

Page 21: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

“Thisaccidentinves=ga=onrevealedastrikinglackofawarenessamongdesigners,contractors,managers,andoverseersaboutthenatureandperformanceofpolymeradhesives,evenasthoseadhesiveswerebeingapprovedforuseanapplica=onswhereafailurewouldpresentanimmediatethreattothepublic.

Evenaderbeingpresentedwithevidenceofanchorcreep,projectmanagersandoverseersfailedtorecognizetheinherentweaknessintheepoxyadhesive–aweaknessthatcouldnotbeovercomeevenwiththebestinstalla=onprac=cesorthemostrigorousshort-termprooftes=ng.”

NTSBAccidentReport

TheUnknownKnown(cont)

Page 22: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

CogniGve Dissonance

From Wikipedia:

•  A psychological term describing the uncomfortable tension that may result from having two conflicting thoughts at the same time, or from engaging in behavior that conflicts with one's beliefs, or from experiencing apparently conflicting phenomena.

•  In simple terms, it can be the filtering of information that conflicts with what

you already believe, in an effort to ignore that information and reinforce your beliefs.

Wikipedia

Page 23: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Accidents related failures of imagination

Failure of imagination has been invoked in regards to the Apollo 1 fire by astronaut Frank Borman in 1967 when he spoke at the Apollo 1 investigation hearings. It has also been mentioned in reference to design flaws in the RMS Titanic and … the failure of the United States to anticipate the attack on Pearl Harbor.

Failure of imagination From Wikipedia: A failure of imagination is a circumstance wherein something seemingly predictable (particularly from hindsight) and undesirable was not planned for.Failure of imagination is related to unknown unknowns.

Page 24: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

ContribuGng Causal Factor

- Inadequate Industry Standards -

ICCAC58:

Eitheradesignsafetyfactorof5.33ora120-daycreeptestisrequiredforFastSetepoxy.

“Given that the ability to sustain a load over a period of time is a typical requirement for almost any type of fastener, the Safety Board is concerned that the ICC has allowed creep testing of epoxy adhesives to be optional. A design engineer should be provided with all of the relevant information about a product before it is used in a safety critical application.” NTSB Accident Report

Page 25: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Consequently…

Tosupportproductqualifica=on,thesupplierprovidedanEvalua=onReport(ER)whichincludedbondstrengthtablesspecifyingasafetyfactorof5.33forFastSetepoxy-nottheresultsofcreeptests*.

*TheSafetyBoardlearnedduringtheinves=ga=onthatFastSetepoxyhadbeentestedforcreep

performancein1995and1996andhadfailedtomeetthestandard

Page 26: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

26

Page 27: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

27

Page 28: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

DesGnaGon Disaster

OnMarch3,1974,TurkishAirlinesFlight981,onarou=neflightfromParistoLondon,crashedinadenseforestinFrance,resul=nginthelossofall346personsaboard.At11,500feet,thedifferen=alpressureinthecabincausedtheadcargodoortoopenandbeblownoff.Thelargeholesuddenlyappearinginapressurehullcreatedanoutwardaccelera=onofairsorapidastoresembleabombexplosion.

Theexplosiondestroyedtheflooringabovethecargohold,severingthecontrolcablesfortherudder,theelevators,andthenumbertwoengine.

28

The proximate cause of the accident was determined to be a faulty latch.

Page 29: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

29

Page 30: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

30

LatchDesign

“Thereweremul=plecomplexlinkagesbetweentheexternalhandleandthelockingpinbarwhich,inaggregate,werefartooweakandflexible.Ratherthanencounteringanirresis=bleforceifthelockingpinshitthelugsofanunclosedlatch,abaggagehandlerofnormalstrengthcouldpushthehandlefullydown,thinkingthathehadthusinsuredtheclosingofthedoorwhenallhehaddonewasbendtheinternalbarsandrodsoutofshape.”

Page 31: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

31

“Itwas,byanysenseofsafetyengineering,agimcrackpieceofdesign.Yet,becauseofdecisionstakenaboutfloorstrengthandcontrol-cableroutes,thesafetyofeveryman,womanandchildwhowentaboardtheDC-10wasdependentupontheefficacyofthelinkagesfromthemomenttheplanewentintoservice.”

PaulEddy

Des=na=onDisaster

Page 32: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

32

Contribu=ngCausalFactors DesignChoices

•  DoorConfigura=on•  Rou=ngofcables,hydrauliclines&wire•  Floorstrength•  Latchdesign•  Cockpitindicatorlight•  Ventdoordesign

Page 33: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

33

FMEA

In the summer of 1969, Douglas asked Convair to draft a FMEA for the lower cargo door system of the DC-10. Convair produced a document which accurately foresaw the deadly consequences of a cargo-door latch failure. But neither Convair’s draft FMEA, nor anything closely resembling it, was ever shown to the FAA.

Page 34: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

The Gi;

OnMay29,1970,duringgroundtes=ngofShip1toprepareitforitsupcomingmaidenflight,theaircondi=oningsystemwasbeingexercisedtobuildupapressuredifferen=alof4to5poundspersquareinch.Suddenly,theforwardlowercargodoorblewopencausingalargesec=onofthecabinfloortocollapseintothehold.

McDonnellDouglasaBributedtheincidentalmosten=relytohumanfailureonthepartofthebaggagehandler.

34

Page 35: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

35

TheApplegateMemorandum:

“Thepoten=alforlongtermConvairliabilityhasbeencausingmeincreasingconcernforseveralreasons…theairplanedemonstratedaninherentsuscep=bilitytocatastrophicfailurewhenexposedtoexplosivedecompressionofthecargocompartmentin1970groundtests…Itseemstomeinevitablethatinthetwentyyearsaheadofus,DC-10cargodoorswillcomeopenandcargocompartmentswillexperiencedecompressionandIwouldexpectthistousuallyresultinthelossoftheairplane.”

F.D.Applegate DirectorofProductEngineering

Convair

Page 36: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

The Gi; (Part 2)

OnJune12,1972,AmericanFlight96departedDetroit,MIandwasclimbingthrough11,750feetwhentherearcargodoorblewoutcausinganexplosivedecompressionandlossofflightcontrols.ThecrewmanagedtoregaincontroloftheplaneandreturntoDetroit.

“Thedesigncharacteris=csofthelatchingmechanismpermiBedthedoortobeapparentlyclosed,when,infact,thelatcheswerenotfullyengagedandthelockpinswerenotinplace.” Na=onalTransporta=on

SafetyBoardAccidentReport(NTSB) February28,1973

36

Page 37: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

TheMidnightGentlemen’sAgreement:

ThepresidentofDouglasspersuadedtheFAAAdministratorthatcorrec=vemeasurescouldbeundertakenasaresultofagentleman’sagreement,therebynotrequiringtheissuanceofanFAAAirworthinessDirec=ve.

Page 38: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

“ When you have a well–constructed state with a

well-framed legal code, to put incompetent officials in charge of administering the code is a waste of good laws, and the whole business degenerates into farce.”

Plato Laws (Book IV)

Page 39: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

PertheGentlemen’sAgreement,Douglasissuedtwoairlineservicebulle=ns:

1.  Installapeepholeandadecalshowingdiagramma=callywhatthehandlerwouldseeifthelockingpinwassafelyhome.IssuedasaSafetyAlert.

2.Installasupportplatetoholdupthetorquetubejustinsidethehandle.Issuedasarou=neservicebulle=n.

Page 40: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

PlanningDepartmentrecordsclearlyshowthatonJuly18,1972,threeinspectorsseeminglyappliedstampsindica=ngthatthesupportplatehadbeeninstalledandthelocktubehadbeenmodified.Thesethreemenwerebroughtforwardandexaminedunderoath.ItemergedthatnotoneofthesethreecouldrecallhavingworkedonthecargodoorofanyDC-10atany=me.Norcouldtheyrecallonanyoccasionwhateveronwhichtheyhadworkedtogether.

Douglasmaintainedtotheendthathumanerrormustaccountforthefalsityoftherecords.

40

A Clear Case of Fraud

Page 41: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

“Becausehistoryisanunrepeatableexperiment,wecannotprovethattheextraurgency,legalweight,andpublicitywhichgowithAirworthinessDirec=veswouldnecessarilyhavemadethedifference.Butthecrucialpointisthedetermina=onontheFAAAdministrator’spartthattheDouglascompanyitselfcouldbeledtohandlethemaBerinitsownway.”

PaulEddy

Des=na=onDisaster

Page 42: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Apollo1CommandModule

FirstinItsClass

Itwasmuchlargerandmorecomplexthananypreviousdesign:theleadingedgeofU.S.

spacecradtechnology.42

The Loss of Two Technological Marvels with All Crew Aboard

USSThresher

FirstinItsClassItwasfast,quietanddeepdiving:theleading

edgeofU.S.submarinetechnology.

Page 43: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

OnApril10,1963,whileengagedinadeeptestdive220mileseast

ofCapeCod,MA,theUSSThresher

waslostatsea,seBlingatadepth

of8,400feetwithallaboard.

Intheend,112navalofficersand

enlistedpersonneland7civiliansperished.

43

Loss of the USS Thresher

Page 44: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Loss of the Apollo 1 Command Module

OnJanuary27,1967,theApollo1crewenteredthespacecradtoperformalaunchcountdownrehearsaltest.Thetestcommencedwithinstallingthehatchdoorandpurgingthecabinwitha100percentoxygenatmosphere.

Pad34:SiteoftheApollo1Fire

Hourslater,asparkfromfaultywiringinsula=onignitedafireconsuminganabundanceofflammablematerialsinthecockpit.Thefirecreatedanoverwhelmingpressureagainstthehatchdoor,sealingthecrewinside.

Grissom,White,andChaffee

44

Page 45: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

USSThresher

Improperlybrazedpipejoint

Apollo1

“Theboardfoundnumerousexamplesinthewiringofpoorinstalla=onand

poorworkmanship.”

PoorlybrazedpipesledtotheelectricalshortagethatledtothelossoftheUSSThresher

Wireswherethefirewassuspectedtohavestarted

45

ContribuGng Causal Factors Inadequate Workmanship

Page 46: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Apollo1KennedySpaceCenterinspectorscitedmul=pleinstancesofdeficientparts,equipment,andworkmanship.

46

ContribuGng Causal Factors IneffecGve CorrecGve AcGon

USSThresherPortsmouthNavalShipyardinspectorsusingnewlydevelopedultrasonictes=ngtechniquesiden=fiednumerousinstancesoffaultybrazedjoints;however,manybrazedjointsontheUSSThresherwerenevertestedusingthenewtechnique.

Page 47: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Apollo1•  Pureoxygenatmosphere

•  Combus=blematerials

•  Inwardopeninghatch•  Inadequateescapeprovisions

47

ContribuGng Causal Factors

Vulnerable Design Inadequate Emergency Recovery

Unforeseen Failure Mode

USSThresher•  Reactorshutdown•  Impairedaccesstovitalequipment

•  Compromisedballasttankblow

WreckagefromtheUSSThresher’ssonardomecanbeseenontheoceanfloor

Page 48: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Fallen Astronauts

Thereasoningbehindtheuseofpureoxygenseemedsoundenough.Inthevacuumofspacethecabinpressureneededtobemaintainedatlessthan6psi.Testshadalreadyprovedthatanyfireatthispressure,eveninapureoxygenenvironment,couldbeeasilycontainedandex=nguished.Duringgroundtests,however,asea-levelpressureof14.7psiwouldenvelopthespacecrad.Iftheoutsidepressureexceededthatinsidethespacecradbymorethan2psi,therewasachancethatthepressurehullcouldrupture,soitwasimpossibletotestthespacecradonthepadusingthe5.2to5.6psithatwouldbestandardoncethespacecradhadachievedorbit.Insteadtheengineerscrankeduptheinteriortomorethan16psitoexceedsea-levelpressure.Itwouldprovetobeafatalerrorofreasoning:thateasilycontainedfireinspaceatlessthan6psiwouldbecomeanexplosiveinfernoat16psi.

ColinBurgess

Page 49: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

49

When VulnerabiliGes Line Up, Consequences Can Be DevastaGng

Page 50: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

SapaProfilesmayhavealteredmechanicalproper=estestresultsperformedonaluminumextrusionsproducedfromatleast1996to2007,andmayhavereportedalteredtestreportsinmaterialcer=fica=onsgiventoitscustomersduringthat=meframe.

50

Fraud

Page 51: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

51

Counterfeit Parts

Page 52: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

CombaXng Fraud Heightened Awareness and Understanding

Supplier Oversight and TesGng of Incoming Product

Western Titanium

M&M Metals A&P Alloys

52

Page 53: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

“OrbitalATKandNASAdiscoveredthepresenceofadefectintroducedduringmachiningofthebearingborehousing…Forensicinves/ga/onofEngineE17,whichfailedduringATPinMay2014,discoveredthepresenceofasimilarnon-conformingdefectinthehousing”

53

Workmanship/CorrecGve AcGon

Page 54: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

JULY 20, 2015

CRS-7 INVESTIGATION UPDATE “Preliminaryanalysissuggeststheoverpressureeventintheupper-stageliquidoxygentankwasini=atedbyaflawedpieceofsupporthardware(a“strut”)insidethesecondstage.SeveralhundredstrutsflyoneveryFalcon9vehicle,withacumula=veflighthistoryofseveralthousand.Thestrutthatwebelievefailedwasdesignedandmaterialcer=fiedtohandle10,000poundsofforce,butfailedat2,000pounds,afive-folddifference.Detailedclose-outphotosofstageconstruc=onshownovisibleflawsordamageofanykind.”

54

Material Control

Page 55: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Muskalsoblamesoverconfidencefortheaccident…“Whenyou’veonlyeverseensuccess,youdon’tfearfailurequiteasmuch.

Page 56: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

Highperformingorganiza/onsarepreoccupiedwiththeprospectoffailure.

Page 57: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

"What we really learned from the Apollo fire, in the words of [former astronaut] Frank Borman, was the failure of imagination," said William H. Gerstenmaier, NASA's associate administrator for space operations. "We couldn't imagine a simple test on the pad being that catastrophic. "The message to the team is to remember how difficult our business is, the importance of staying focused and using our imaginations to envision what can go wrong."

40 years later, recalling the lessons of Apollo 1 January 28, 2007|Michael Cabbage | Orlando Sentinel

Page 58: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

AddiGonal Stuff

58

Page 59: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

DemonstratedReliabilityvsInsight/Oversight

Page 60: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

USS IWO JIMA Mishap

A Set-up for Failure

BryanO’ConnorChief,OfficeofSafetyandMissionAssuranceNASAHeadquarters,Washington,DCOSMAPOC,BrianHughiB(202)358-1572

Page 61: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

(61)

What Happened

-October1990:USSIWOJIMAAmphibiousAssaultShipDeployedtoPersianGulf,Opera=onDESERTSHIELD

-DockedatBahrainshipyardforemergentrepairs

-Asshipwasleavingport-onehouraderpropulsionplantbroughtonline-bonnetfastenersfora4”valvesupplyingsteamtoShipsServiceTurbineGeneratorfailedcatastrophically

-  850degreesuperheatedsteamat600psiescapedintomannedcompartment

-  Ninesailorskilledinstantly,onemorefatallyinjured

Page 62: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

(62)

Proximate Cause

Unauthorizedsubs=tu=onofblackoxidecoatedbrassfastenersforhighstrengthsteelfasteners

AlloySteelNutsBlackOxideCoatedNuts

Page 63: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

(63)

The Set-Up

-  BOCBF’svirtuallyiden=calinappearancetohighstrengthsteelfasteners.Coa=ngservednofunc=onalpurpose-appliedinordertostandardizecommonparts

-  Fastenersreadilyavailableandeasilyinterchangeable

-  Brassexhibitssignificantlydiminishedstrengthproper=esfromsteelunderelevatedtemperatures

-  ManufacturerLogoonBOCBFbox:“FastenersforHighStrengthApplica=ons”

Page 64: Common Threads Among Catastrophic Mishaps · Common Threads Among Catastrophic Mishaps Lessons Not Learned Vulnerable Design Workmanship Shortcomings Process Control Failures Failure

(64)

ContribuGng Causal Factors

-  Repairspecifica=onsdidnotiden=fyfastenerpartnumbers

-  NoevidencethatGovernmentholdpointinspec=onswereperformed

-  InadequateknowledgeofLevelImaterialcontrolrequirements

-Segrega=onfromnon-LevelI-Segrega=onoflook-alikeparts-Markings-Colorcoding-Documenta=on