COMPARATIVEREPORT,Volume1
U.S.JHIMDATDatatoU.S.JHSATData
U.S.JointHelicopterImplementationMeasurementDataAnalysisTeam
To
TheUnitedStatesHelicopterSafetyTeam
March2014
TheU.S.JHSATanalyzed523U.S.registeredhelicopteraccidentsthatoccurredinCY2000,CY2001andCY2006(JHSATdata).TheU.S.JHIMDATanalyzed415U.S.registeredhelicopteraccidentsthatoccurredinCY2009‐2011(JHIMDATdata).ThisreportcontainsthecomparativeJHIMDATdatatoJHSATdataanalysis.Thepurposewastoidentifydifferencesofstatisticalsignificance.Bynotingareasofimprovement,regression,andstagnation,thisanalysisallowedtheopportunityforastrategicapproachtofuturehelicopteraccidentratereductionefforts.
1
TableofContents
U.S.JHIMDATMembers.........................................................................................................................................................3
Description.................................................................................................................................................................................4
Industry.......................................................................................................................................................................................6
Activity.........................................................................................................................................................................................7
IndustryandActivityPairs..................................................................................................................................................8
OccurrenceCategory............................................................................................................................................................10
Sub‐OccurrenceCategory...................................................................................................................................................12
JointAircraftSystem/Component(JASC)Codes......................................................................................................14
InitiatorofSystemComponentFailure(SCF)............................................................................................................15
GeneralAccidentCharacteristics....................................................................................................................................17
ProportionsofFatalandNon‐FatalAccidents..........................................................................................................18
AccidentsbyPilot’sFlightHours....................................................................................................................................19
SummaryandConclusions................................................................................................................................................22
References................................................................................................................................................................................24
2
ListofTables
Table1.IndustryComparison...........................................................................................................................................6Table2.ActivityComparison.............................................................................................................................................7Table3.IndustryandActivityPairsComparison.....................................................................................................9Table4.OccurrenceCategoryComparison...............................................................................................................11Table5.Sub‐OccurrenceCategoryComparison......................................................................................................13Table6.JASCCodes..............................................................................................................................................................14Table7.InitiatorofSystemComponentFailures(SCFs).....................................................................................16
ListofFigures
Figure1.DistributionofAccidents‐Pilots'Make/ModelFlightHours.........................................................20
3
U.S.JHIMDATMembersThefollowingJHIMDATMemberorganizedthedata,conductedthecomparativeanalysis,andwrotethecomparativereportbetweentheJHIMDATandJHSATdata.
Name Company/OrganizationLeeRoskop FAARotorcraftStandardsStaff
ThefollowingJHIMDATMemberscontributedtooneorallofthefollowingactivities:analysisoftheJHIMDATdata,reviewofthecomparativereport,anddevelopmentofsummaryandconclusions.
Name Company/Organization PositionSteveGleason SikorskyAircraftCorporation JHIMDATCo‐Chair(Industry)ScottTyrrell FAARotorcraftStandardsStaff JHIMDATCo‐Chair(Government)EricBarnett FAARotorcraftStandardsStaff MemberScottBurgess Embry‐RiddleAeronauticalUniversity MemberTomCaramancio BoeingMilitaryAircraft MemberMarkColborn DallasPoliceDepartment MemberMunroDearing NASA MemberRoyFox BellHelicopterTextron‐Retired MemberGaryHowe BellHelicopterTextron MemberLeeRoskop FAARotorcraftStandardsStaff MemberBillWallace BillWallace&Associates Member
4
DescriptionIn2006,theU.S.JointHelicopterSafetyAnalysisTeam(JHSAT)begananalysisworkonU.S.accidentsthatoccurredincalendaryear2000.Afterpublishingareportwiththeresultsoftheirwork,theJHSATproceededtocompletecomparableanalysisontheU.S.accidentsthatoccurredin2001andfollowedcompletionofthatprojectwithanalysisof2006.In2011,theJHSATconsolidatedtheresultsofallthreerespectiveyearsintoacompendiumreport(hereafterknowninthisreportas“JHSATdata”).TheJHSATdataestablishedabaselineforfutureU.S.helicopteraccidentanalysis.
Asafollowuptothecompendiumreport,theU.S.JointHelicopterImplementationMeasurementandDataAnalysisTeam(JHIMDAT)attemptedtomeasureprogressbycomparingtheJHSATdatatoanalysisperformedonthethreecalendaryears2009‐2011(hereafterknowninthisreportas“JHIMDATdata”).AnalysisoftheJHIMDATdataremainedconsistentwiththeJHSAT’smethodology.However,theextensivedepthofanalysisperformedontheJHSATdatarequiredmonthlymeetingsofoneweekdurationthatwerenolongerfeasibleforanalysisoftheJHIMDATdata.TheJHIMDATtransitionedtoquarterlymeetingsconsistingofamuchsmallerteamcompositionthantheoriginalJHSAT.Toaccountfortherealitiesofasmallerteamthatmetlessfrequently,theJHIMDATreducedthescopeoftheanalysiseffort.Theteamaccomplisheda“highlevel”analysisofthethreeyearsfrom2009‐2011throughquarterlymeetingsconductedovera9monthperiod.
TheJHIMDAT’spreferencewouldhavebeentouseratebasedcomparisonsintheiranalysis.Theratewouldhavebeencalculatedusingaccidentcountsasthenumeratorandflighthoursasthedenominator.However,whileitwaspossibletomakesomereasonableestimatesforthetotalU.S.rotorcraftflighthours,itwasnotpossibletoaccuratelyrefinethatflighthourestimateacrossthenumerousdifferentanalysisareasusedbytheJHIMDAT.Withthislimitation,theJHIMDATmadecomparisonsintheirresearchbasedonaccidentcountsthatwereconvertedtopercentagestoexpressfrequency.
TheJHIMDAT’sintentwastofindwheremeaningfuldifferencesexistedbetweentheJHIMDATdataandtheJHSATdata.Areaswhereimprovementoccurredwerecandidatesforexploringsuccessfulimplementationmeasuresforuseinotherareaswhereprogresswaslesssubstantial.Areaswhereregressionoccurredorstagnationwasevidentwerecandidatesforadaptingdifferentinterventionstrategiestoencourageimprovement.Ineithercase,adjustmentscouldbemadetofurthertheprogresstowardthe80%accidentratedecreasesoughtbytheIHST.InitialcomparisonoftheJHIMDATdataandJHSATdatashoweddifferencesdidexist.However,theJHIMDATsoughttofurtheridentifyareaswherestatisticallysignificantdifferencesexisted.Theteamoptedforachisquaredanalysistosupportthispurpose,basedontheJHSAT’sandJHIMDAT’smethodoforganizingtheaccidentanalysisdataaccordingtoanominalscale.“Nominalscale”describesagroupingofdataintovariouscategoriesbasedonsomecommoncharacteristic.Apartiallistofexampleswouldincludethefollowing:1)“Industry”,suchasPersonal/Private,Instructional/Training,orAerialApplication;2)“Activity”descriptions,suchasPositioning/RTB,Instructional/Training(Dual),orMaintenance/Testflight;and3)“OccurrenceCategories”,suchasLossofControl,Autorotation,orSystemComponentFailure.
5
Thechisquaredanalysisconsistedofeithera“goodnessoffit”test(mostcases)ora“testofhomogeneity”(afewcases).ThegoodnessoffittestallowedtheteamtodirectlycompareonegroupofJHIMDATdatatothecorrespondinggroupofJHSATdatatoassessstatisticaldifferences.Thetestofhomogeneitywasaslightlymorecomplexapproachthattheteamusedforafewlimitedcaseswherethegoalwastoexaminetwocategories(e.g.,VMCorIMC)againsttwoothercategories(e.g.,fatalaccidentornon‐fatalaccident).ThetestofhomogeneitydidnotallowthesamedirectcomparisonofJHIMDATdatatoJHSATdataaswaspossiblewiththegoodnessoffittest.However,aviablecomparisonwasstillpossibleinthesecasesaccordingtothefollowingmethod.TheteamfirstappliedthetestofhomogeneitytoidentifywhetherastatisticaldifferenceexistedwithintheJHSATdata,thenappliedthesametestofhomogeneitytothecorrespondingcategorieswithintheJHIMDATdata,andconcludedbyassessingiftheresultsbetweenthetwodatasetsweresimilarordifferent.Alsoaspartofthechisquaredanalysis(forbothgoodnessoffittestandtestofhomogeneity),theteamoftenusedthestandardizedresidualtodeterminewhatspecificnominaldatawasamajorcontributortodrivingthestatisticaldifference.Forexample,ifastatisticaldifferencewasfoundintheareaofindustrysectors,thestandardizedresidualwouldidentifythespecificindustrysectorsmostresponsibleforthestatisticaldifference.
Foreachanalysisscenariointhisreport,theJHIMDATusedthechisquaredcriticalvaluesassociatedwithp=.01.The“pvalue”isassociatedwiththeprobabilityofrejectinganullhypothesisthatisactuallytrue.Thepvaluesassociatedwithmeasurementsofstatisticalsignificancearetypically.05orless,sotheJHIMDAT’suseofp=.01wasconsistentwithusingarigorousstandardtoassessstatisticalsignificance.InthecaseoftheJHIMDATandJHSATdatacomparison,thenullhypothesisineachcasewasthattherewasnotadifferencebetweenthenominaldatafromtheJHIMDATascomparedtothenominaldatafromtheJHSAT.Inordertorejectthenullhypothesisforanyoftheareasstudiedinthisreport,thecalculatedchisquaredvaluehadtoexceedthechisquaredcriticalvalueforp=.01.AnytimetheteamfoundastatisticaldifferenceintheJHIMDATandJHSATdata,useofp=.01meantthattherewasa1%(orless)probabilitythatourconclusionwasincorrectandastatisticaldifferencebetweenthedatadidnotexist.
AgoaloftheJHIMDATwastoproduceareportthatwassuccinctandeasilyreadsothatresultsfromthereportcouldbequicklytranslatedbythereadersintoimplementation.Wesoughttoavoidacumbersomeandoverlydetaileddocumentthatwouldneverberead.Consistentwiththisapproach,theteamseparatedthecomparativeanalysisintotwovolumes.TheintentwastoensureVolume1emphasizedthemostimportantaspectsofthecomparativeanalysis.TheteamintentionallytruncatedmostofthetablesinVolume1suchthatonlythemostfrequentlycitedareasintheanalysiswerelisted.Thedatasuggestedtheseareascouldhavethemostsignificantimpactonfutureaccidentreductionefforts.Volume2ofthecomparativeanalysishasthecompletedatatablesforthereaderwhowouldliketoexploretheanalysisatanadditionallevelofdetail.Volume2alsoincludessomedatatablesthatwerelimitedtoatextualsummaryinVolume1.
6
IndustryIndustrydescribedthetypeofworkoroperationwherethehelicopteristypicallyused.TheteamcategorizedeachaccidentfromboththeJHSATdataandJHIMDATdatabyIndustryintooneoffifteenpossibleoptions.Table1liststhefivemostfrequentlycitedIndustrycategoriesfromtheJHIMDATdataalongsidetherespectivepercentagesfromtheJHSATdata.TheIndustrycategoriesarelistedindescendingorderoffrequencybasedontheJHIMDATcolumn.
ThereweretwoseparatechisquaredanalysesaccomplishedasrelatedtoIndustry.
1) TheteamanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyIndustryagainsttheaggregategroupofJHSATaccidentscategorizedbyIndustry.Theconclusionwasthetwodatasetswerestatisticallydifferent.MajorcontributorstothestatisticaldifferenceweretheincreaseintheproportionofAerialApplicationaccidentsandthedecreasesintheproportionsofFirefighting,Logging,andElectronicNewsGatheringaccidents.
2) Theteamconductedaseparate,individualanalysisofeachspecificIndustrysegmentcomparingthetwodatasets.AsshowninTable1,AerialApplicationwastheonlyoneofthefivemostfrequentlyoccurringIndustrycategorieswhereastatisticallysignificantdifferenceexistedbetweentheJHSATdataandtheJHIMDATdata.Inthatparticularcase,theteamfoundastatisticallysignificantincreaseinaccidents.
Table1.IndustryComparison
20.7% (86) 18.5% (97)20.5% (85) 17.6% (92)15.7% (65) 10.3% (54)8.0% (33) 7.6% (40)7.5% (31) 7.5% (39)
DenotesstatisticallysignificantincreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesstatisticallysignificantdecreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesnostatisticallysignificantchangeinproportionofaccidentsfromJHSATdatatoJHIMDATdata
AerialApplication
CommercialEmergencyMedicalServices
JHIMDAT(CY09‐11):415Accidents JHSAT(CY00‐01,06):523AccidentsIndustry
Personal/PrivateInstructional/Training
7
ActivityActivityisdifferentfromIndustryinthatitdescribedthespecificfunctionthehelicopterwasengagedinatthetimetheaccidentactuallyoccurred.TheteamcategorizedeachaccidentfromboththeJHSATdataandJHIMDATdatabyActivityintooneoftwenty‐sevenpossibleoptions.Table2liststhefivemostfrequentlycitedActivitycategoriesfromtheJHIMDATdataalongsidetherespectivepercentagesfromtheJHSATdata.TheActivitycategoriesarelistedindescendingorderoffrequencybasedontheJHIMDATcolumn.
ThereweretwoseparatechisquaredanalysesaccomplishedasrelatedtoActivity.
1) TheteamanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyActivityagainsttheaggregategroupofJHSATaccidentscategorizedbyActivity.Theconclusionwasthetwodatasetswerestatisticallydifferent.AmongsomeofthemajorcontributorstothestatisticaldifferencewastheincreaseintheproportionofaccidentsinthefollowingActivities:Instructional/Training(Dual),Personal/Private,andInstructional/Training(Solo).
2) Theteamconductedaseparate,individualanalysisofeachspecificActivitycomparing
theJHIMDATdatatotheJHSATdata.TheresultsforthefivemostfrequentlyoccurringActivitiesareinTable2.BothInstructional/Training(Dual)andPersonal/Privateshowedstatisticallysignificantincreases.
Table2.ActivityComparison
19.3% (80) 14.0% (73)17.8% (74) 12.4% (65)13.7% (57) 13.2% (69)8.9% (37) 8.0% (42)5.3% (22) 3.3% (17)
JHIMDAT(CY09‐11):415Accidents
JHSAT(CY00‐01,06):523AccidentsActivity
Instructional/Training(Dual)Personal/Private
DenotesnostatisticallysignificantchangeinproportionofaccidentsfromJHSATdatatoJHIMDATdata
Positioning/RTBAerialApplication‐Spraying/DisbursingInstructional/Training(Solo)
DenotesstatisticallysignificantincreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesstatisticallysignificantdecreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
8
IndustryandActivityPairsThecombinationoftheIndustrycategoryforeachaccidentandtheActivitycategoryforeachaccidentcanbepairedtogethertorefinetheunderstandingoftheaccidentsthathappened.Thispairingresultedinninety‐sevenIndustryandActivitycombinationsthatwereusedintheJHIMDATandJHSATanalysis.Table3liststhethirteenmostfrequentlyoccurringIndustryandActivitypairsfromtheJHIMDATdataalongsidetherespectivepercentagesfromtheJHSATdata.TheIndustryandActivitypairsarelistedindescendingorderoffrequencybasedontheJHIMDATcolumn.
TheteamaccomplishedtwoseparatechisquaredanalysesasrelatedtoIndustryandActivitypairs.
1) WeanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyIndustryandActivitypairsagainsttheaggregategroupofJHSATaccidentscategorizedbyIndustryandActivitypairs.Theconclusionwasthetwodatasetswerestatisticallydifferent.AmongsomeofthemajorcontributorstothestatisticaldifferencewastheincreaseintheproportionofaccidentsinthefollowingIndustryandActivitypairs:Personal/Private–Personal/Private,Instructional/Training‐Instructional/Training(Solo),andAerialApplication–Positioning/RTB.
2) Theteamattemptedaseparate,individualanalysistocomparethedatasetsforeachofthetopthirteenIndustryandActivitypairs.However,anaccurateindividualanalysiswasonlypossibleforsevenofthethirteenpairs.Fortheremainingsixpairs,somecalculationsassociatedwiththechisquaredanalysisinthesecasesresultedinnumbersbelowtheminimumacceptablevaluesnecessarytoensureanaccuratestatisticalcalculation.ThesixIndustryandActivitypairsexcludedfromtheanalysisareannotatedaccordinglyinTable3.TheonlystatisticaldifferenceamongthesevenpairsthatwereindividuallyanalyzedwasastatisticallysignificantincreaseinPersonal/Private–Personal/Private.
ThetenthmostfrequentlyoccurringIndustryandActivitypairfromtheJHIMDATdatashowninTable3wasAerialApplication/AerialApplication–Other.Sincethe“Other”designationisnondescriptwiththepotentialforvariousinterpretations,theteamdeterminedthefollowingclarifyingdefinitionwasappropriate.AerialApplication–OtherdescribedanAerialApplicationaccidentwhereitwaseitherunclearwhatactivitythepilotwasengagedinatthetimeofthecrashortheactivitythepilotwasengagedinwassoinfrequentitdidnotwarrantamorespecificActivitycategory.
9
Table3.IndustryandActivityPairsComparison
Activity
Personal/Private 17.8% (74) 12.0% (63)Instructional/Training(Dual) 15.2% (63) 13.6% (71)AerialApplication‐Spraying/Disbursing 8.9% (37) 8.0% (42)Instructional/Training(Solo) 4.3% (18) 2.7% (14)Positioning/RTB 4.1% (17) 5.5% (29)Positioning/RTB 3.4% (14) 1.1% (6) XPassenger/Cargo(Commercial) 2.7% (11) 3.3% (17)Instructional/Training(Dual) 2.7% (11) 0.4% (2) XExternalLoad‐Line 2.4% (10) 2.5% (13)AerialApplication‐Other 2.4% (10) 1.0% (5) XPassenger/Cargo(Commercial) 2.2% (9) 2.1% (11) XEmergencyMedicalServices 1.9% (8) 1.1% (6) XAerialObservation‐Fish/GameSpotting 1.9% (8) 1.0% (5) X
X
ExternalLoadAerialApplication
Instructional/Training
DenotesstatisticallysignificantincreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesstatisticallysignificantdecreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesnostatisticallysignificantchangeinproportionofaccidentsfromJHSATdatatoJHIMDATdata
Excludedfromindividualchisquaredanalysis
JHIMDAT(CY09‐11):415Accidents
JHSAT(CY00‐01,06):523Accidents
Personal/PrivateInstructional/Training
AerialApplication
OffshoreEmergencyMedicalServices
AerialObservation
Industry
EmergencyMedicalServicesAerialApplicationCommercialLawEnforcement
10
OccurrenceCategoryOccurrenceCategoriesprovidedaconcisedescriptionof“whathappened”inanaccident.Table4liststheninemostfrequentlycitedOccurrenceCategoriesfromtheJHIMDATdataalongsidetherespectivepercentagesfromJHSATdata.ThetableliststheOccurrenceCategoriesindescendingorderoffrequencybasedontheJHIMDATcolumn.
AchisquaredanalysisthatcomparedtheaggregategroupofJHIMDATtoJHSATaccidentsbyOccurrenceCategorieswasnotpossibleusingthesameformatastheIndustryandActivitydata.ThisisbecauseeachaccidentwasnotlimitedtoassignmentinasingleOccurrenceCategory.BoththeJHSATandJHIMDATmethodologyallowedeachindividualaccidenttobecodedwithuptofourdifferentOccurrenceCategories.ThisaspectofthemethodologyexplainswhythepercentagesofOccurrenceCategoriesfromeachdatasetshouldnotbesummedwiththeexpectationoftheresulttotaling100%.MultipleOccurrenceCategoriesperaccidentensuredthesummedtotalwillfarexceed100%.
However,theteamwasabletoperformaseparate,individualchisquaredanalysisforeachOccurrenceCategory.Foreachcase,thiswaspossiblebygroupingallaccidentswhereanOccurrenceCategoryhadbeenusedatleastonceintoonesetofdata(e.g.,allLossofControlaccidents),whilegroupingallaccidentswherethatsameOccurrenceCategorywasneverusedintoanothersetofdata(e.g.,allaccidentswhereLossofControlwasnotcited).TheresultsfortheninemostfrequentlyoccurringOccurrenceCategoriesareinTable4.LossofControlandControlledFlightIntoTerrain(CFIT)showedstatisticallysignificantincreaseswhileSystemsComponentFailure(SCF)showedastatisticallysignificantdecrease.
TheteamdecidedfurtherexplanationwasneededforoneoftheOccurrenceCategoriesinTable4.AlthoughAbnormalRunwayContact(ARC)islistedasthethirdmostfrequentlyassignedOccurrenceCategoryintheJHIMDATdata,itwasexcludedfromtheindividualchisquaredanalysis.TheteamassignedARCto24.6%oftheaccidentsintheJHIMDATdataascomparedto7.6%oftheaccidentsintheJHSATdata.ItwouldappeartheOccurrenceCategoryisaclearcandidateforassessingwhetherasignificantstatisticaldifferenceexisted.However,theJHIMDATattributedthisrelativelylargedifferenceinproportionstoadifferentunderstandingandassignmentoftheARCcategoryintheJHIMDATanalysisasopposedtohowitwaspreviouslyusedintheJHSATanalysis.TheJHIMDATdidnotthinktherewasanactualincreaseinthecasesofARC;rather,therewasadifferentapplicationoftheOccurrenceCategorybytheJHIMDAT.ThisledtoinconsistencyinhowitwasassignedintheJHIMDATdatawhencomparedtotheearlierJHSATwork.
11
Table4.OccurrenceCategoryComparison
47.5% (197) 41.5% (217)32.8% (136) 31.7% (166)24.6% (102) 7.6% (40)21.4% (89) 27.5% (144)19.8% (82) 16.4% (86)8.2% (34) 7.6% (40)8.0% (33) 10.7% (56)7.0% (29) 6.1% (32)6.7% (28) 3.1% (16)
X
JHSAT(CY00‐01,06):523Accidents
LOC‐LossofControl
ARC‐AbnormalRunwayContact
VIS‐Visibility
AUTO‐Autorotation
SCF‐SystemComponentFailureSTRIKE
OccurrenceCategory JHIMDAT(CY09‐11):415Accidents
FUEL
AbnormalRunwayContactexcludedfromindividualchisquaredanalysis
FIRECFIT‐ControlledFlightintoTerrain
DenotesstatisticallysignificantincreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesstatisticallysignificantdecreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesnostatisticallysignificantchangeinproportionofaccidentsfromJHSATdatatoJHIMDATdata
12
Sub‐OccurrenceCategoryTheSub‐OccurrenceCategoryservedthepurposeofprovidingamoredetaileddescriptionwithineachOccurrenceCategory.Table5liststheelevenmostfrequentlyassignedOccurrenceCategory/Sub‐OccurrenceCategorypairsfromtheJHIMDATdataalongsidetherespectivepercentagesfromJHSATdata.ThetableliststheOccurrenceCategory/Sub‐OccurrenceCategorypairsindescendingorderoffrequencybasedontheJHIMDATcolumn.
ForthesamereasonsnotedintheOccurrenceCategoryportionofthereport,achisquaredanalysisthatcomparedtheaggregategroupofJHIMDATtoJHSATaccidentsbyOccurrenceCategory/Sub‐OccurrenceCategoriespairswasnotpossible.
However,theteamdidperformaseparate,individualchisquaredanalysisforeachOccurrenceCategory/Sub‐OccurrenceCategorypair.Foreachcase,thiswaspossiblebygroupingallaccidentswhereapairhadbeenusedatleastonceintoonesetofdata,whilegroupingallaccidentswherethatsamepairwasneverusedintoanothersetofdata.TheresultsfortheelevenmostfrequentlycitedOccurrenceCategory/Sub‐OccurrenceCategorypairsareinTable5.Fourofthetopelevenmostfrequentlycitedpairsshowedastatisticallysignificantincreaseintheproportionofaccidents.SystemComponentFailure–HelicopterwastheoneIndustry/Activitypairthatshowedimprovementasevidentbythestatisticallysignificantdecreaseinaccidents.
AsalsonotedintheOccurrenceCategorysectionofthereport,theteamdidnotanalyzetheSub‐OccurrenceCategoryofAbnormalRunwayContact(ARC)forastatisticaldifference.Theexplanationisthesameaspreviouslynoted.TheteamdidnotthinktherewasanactualincreaseinthecasesofARC;rather,therewasadifferentapplicationoftheOccurrenceCategorybytheJHIMDATandthisledtoinconsistencyinhowitwasassignedbytheJHIMDATwhencomparedtotheearlierJHSATwork.
13
Table5.Sub‐OccurrenceCategoryComparison
24.6% (102) 7.6% (40) X22.9% (95) 21.8% (114)21.7% (90) 15.1% (79)11.8% (49) 10.9% (57)8.7% (36) 5.5% (29)8.7% (36) 4.0% (21)8.4% (35) 5.9% (31)8.0% (33) 10.1% (53)7.5% (31) 14.3% (75)6.7% (28) 3.1% (16)6.0% (25) 5.2% (27)
X
LossofControl‐PerformanceManagement
OccurrenceCategory/Sub‐OccurrenceCategory JHIMDAT(CY09‐11):415Accidents
JHSAT(CY00‐01,06):523Accidents
AbnormalRwyContact‐AbnormalRwyContactAutorotation‐Emergency
Autorotation‐PracticeStrike‐LowAltitudeMissionStrike‐TakeofforLandingLossofControl‐DynamicRollover
Fire‐PostImpact
DenotesstatisticallysignificantdecreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesnostatisticallysignificantchangeinproportionofaccidentsfromJHSATdatatoJHIMDATdata
AbnormalRunwayContactexcludedfromindividualchisquaredanalysis
SystemComponentFailure‐EngineSystemComponentFailure‐HelicopterCFIT‐CFIT
DenotesstatisticallysignificantincreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
14
JointAircraftSystem/Component(JASC)CodesTheoriginalJHSATdatadelineatedSystemComponentFailure(SCF)accidentsintooneoffourcategories:1)Engine,2)Helicopter,3)MissionEquipment,or4)Unconfirmed/Perceived.TheJHIMDATsoughttoexpandonthespecificityassociatedwithSCFaccidentanalysisthroughuseofJASCcodes.TheteamincludedaJASCCodeanalysisforthe89SCFaccidentsfromtheJHIMDATdata(comprising21.4%ofthe415totalaccidents)andthenreturnedtotheoriginalJHSATdataanddidlikewiseforthe144SCFaccidents(comprising27.5%ofthe523totalaccidents).Theteamusedthirty‐ninedifferentfourdigitJASCCodesintheJHIMDATdataandfifty‐sixdifferentfourdigitJASCCodesintheJHSATdata.
BecauseofthemanydifferentJASCCodesusedbetweenthetwodatasets,theteamhadtoconsolidatetheminordertoaccomplishanaccuratestatisticalcomparisonbetweentheJHIMDATandtheJHSATdata.WesortedthevarietyofJASCCodesintothreegeneralcategories.Table6liststheconsolidatedJASCCodecategoriesfromtheJHIMDATdataalongsidetherespectivepercentagesfromtheJHSATdata.TheJASCCodesarelistedindescendingorderoffrequencybasedontheJHIMDATcolumn.
ThereweretwoseparatechisquaredanalysesaccomplishedasrelatedtoJASCCode.
1) TheteamanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyJASCCodeagainsttheaggregategroupofJHSATaccidentscategorizedbyJASCCode.Theconclusionwasthetwodatasetswerenotstatisticallydifferent.
2) Theteamconductedaseparate,individualanalysisforeachoftheJASCCodecategoriesinTable6.EachoftheconsolidatedcategorieswascomparedtoallaccidentswherethatconsolidatedcategoryofJASCCodeswasnotused.AsshowninTable6,therewasnotastatisticallysignificantdifferencebetweentheJHSATdataandtheJHIMDATdataforanyofthethreecategories.
Table6.JASCCodes
55.1% (49) 45.8% (66)29.2% (26) 38.2% (55)15.7% (14) 16.0% (23)
DenotesnostatisticallysignificantchangeinproportionofaccidentsfromJHSATdatatoJHIMDATdata
PowerplantSystems(7000,8000series)
RotorSystems(6000series)
AirframeSystems(2000,3000,4000,5000series)
JHIMDAT(CY09‐11):89SCFAccidents
JHSAT(CY00‐01,06):144SCFAccidents
JASCDescription
DenotesstatisticallysignificantincreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesstatisticallysignificantdecreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
15
InitiatorofSystemComponentFailure(SCF)TheJHIMDATobservedthateachSystemComponentFailure(SCF)hadaninitiatingevent.TheteamdesignatedthiseventastheInitiatorofSCFandfoundthatfourmajorcategoriesexisted:MaintenanceError,ManufacturingError,PilotError,orUnknown.Table7hasthefourcategoriesorganizedindescendingorderoffrequencybasedontheJHIMDATcolumn.IdenticaltotheJASCCodeanalysis,notethatthepercentagesshowninthetablearebasedonlyonthesubsetofSCFaccidents.Thisconsistedof89accidentsfromtheJHIMDATdata(21.4%ofthe415totalaccidents)and144accidentsfromtheJHSATdata(27.5%ofthe523totalaccidents).
ThereweretwoseparatechisquaredanalysesaccomplishedasrelatedtotheInitiatorofSCF.
1) TheteamanalyzedtheaggregategroupofJHIMDATaccidentscategorizedbytheInitiatorofSCFagainsttheaggregategroupofJHSATaccidentscategorizedbytheInitiatorofSCF.Theconclusionwasthetwodatasetswerestatisticallydifferent.MajorcontributorstothestatisticaldifferenceweretheincreaseintheproportionofaccidentswheretheInitiatorofSCFwasUnknownandthedecreaseintheproportionofaccidentswheretheInitiatorofSCFwasManufacturingError.
2) Theteamattemptedaseparate,individualanalysisforeachspecificInitiatorofSCFcomparingtheJHIMDATdatatotheJHSATdata.TheresultsareinTable7.Ofnote,however,isthattheteamhadtoexcludethePilotErrorcategoryfromindividualanalysis.ThiswasnecessarybecausesomecalculationsassociatedwiththechisquaredanalysiswiththePilotErrorcategoryresultedinnumbersbelowtheminimumacceptablevaluesneededtoensureanaccuratestatisticalcalculation.
ThestatisticallysignificantincreaseintheUnknowncategoryoftheJHIMDATdatainhibitedtheabilitytodrawmoremeaningfulconclusionsfromtheanalysis.TheUnknowncategoryeffectivelymasksthetrueproportionsofaccidentsinthemoredescriptivecategoriesassociatedwithInitiatorofSCF.Forexample,notethattheInitiatorofSCFsattributabletoManufacturingErrorshowedastatisticallysignificantdecreasebetweendatasets.However,itispossiblethedecreasewasartificial.WedidnotknowhowmanyoftheaccidentswithintheUnknowncategorymayhaveactuallybeenattributabletoManufacturingErrorifmoreinvestigativeinformationhadbeenavailable.So,ManufacturingErrormayhaveshownatruestatisticallysignificantdecrease,orthedecreasemayhavebeenartificialifmoreManufacturingErrorcaseswerehiddenintheUnknowncategory.ThissameconditioncouldhaveaffectedtheresultsoftheMaintenanceErrorandPilotErrorcategories.
16
Table7.InitiatorofSystemComponentFailures(SCFs)
44.9% (40) 18.1% (26)40.4% (36) 50.7% (73)7.9% (7) 21.5% (31)6.7% (6) 9.8% (14) X
X Excludedfromindividualchisquaredanalysis
PilotError
DenotesstatisticallysignificantincreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesstatisticallysignificantdecreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata
DenotesnostatisticallysignificantchangeinproportionofaccidentsfromJHSATdatatoJHIMDATdata
ManufacturingError
IniatorofSCF JHIMDAT(CY09‐11):89SCFAccidents
JHSAT(CY00‐01,06):144SCFAccidents
UnknownMaintenanceError
17
GeneralAccidentCharacteristicsTheJHIMDATalsocomparedthefollowingtraitsbetweenthetwodatasets:TypeEngineInstalled,AccidentsbyMonth,WeatherCondition,andLightCondition.
TypeEngineInstalledTheteamgroupedaccidentsfromboththeJHIMDATdataandtheJHSATdataintooneofthreecategoriesofinstalledengineorengines:turbinetwin,turbinesingle,orreciprocating.WeanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyTypeEngineInstalledagainsttheaggregategroupofJHSATaccidentscategorizedbyTypeEngineInstalled.Theconclusionwasthetwodatasetswerestatisticallydifferent.Majorcontributorstothestatisticaldifferenceweretheincreaseintheproportionofaccidentswhererotorcrafthadareciprocatingengineinstalledandthedecreaseintheproportionofaccidentswhererotorcrafthadtwinturbineenginesinstalled.AccidentsbyMonthTheteamgroupedaccidentsfrombothdatasetsbasedonthemonthofoccurrence.WeanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbymonthofoccurrencecomparedtotheaggregategroupofJHSATaccidentscategorizedbymonthofoccurrence.Therewasnotastatisticaldifferencebetweenthetwodatasets.Theteamalsoattemptedaseparate,individualanalysisofaccidentsbymonthforeachofthefivemostfrequentlycitedIndustrycategories:1)Personal/Private,2)Instructional/Training,3)AerialApplication,4)EMS,and5)Commercial.However,anaccurateindividualanalysiswasonlypossibleforInstructional/Training.FortheremainingfourIndustrycategories,somecalculationsassociatedwiththechisquaredanalysisforeachcaseresultedinnumbersbelowtheminimumacceptablevaluesnecessarytoensureanaccuratestatisticalcalculation.Instructional/TrainingshowedastatisticallysignificantdifferencefromtheJHSATdatatotheJHIMDATdata.ThemajorcontributorstothestatisticaldifferenceinInstructional/TrainingaccidentsbymonthwereanincreaseintheproportionofaccidentsinOctoberandadecreaseintheproportionofaccidentsinDecember.WeatherConditionTheteamgroupedaccidentsfromboththeJHIMDATandtheJHSATaccordingtowhethertheaccidentoccurredinVisualMeteorologicalConditions(VMC)orInstrumentMeteorologicalConditions(IMC).TherewasnotastatisticallysignificantdifferenceintheproportionofVMCandIMCaccidentsbetweenthetwodatasets.LightConditionTheteamgroupedaccidentsfrombothdatasetsaccordingtowhethertheaccidentoccurredduringthedayorduringthenight.Wecategorizedthefollowinglightconditionsasnightforpurposesoftheanalysis:dawn,dusk,twilight,nightbright,andnightdark.TherewasnotastatisticallysignificantdifferenceintheproportionofdayandnightaccidentsfromtheJHSATtotheJHIMDATdata.
18
ProportionsofFatalandNon‐FatalAccidentsTheteamselectedseveraldifferentdatacategoriestoanalyzewhetherastatisticaldifferenceexistedintheproportionsoffatalversusnon‐fatalaccidentsintheJHIMDATdataascomparedtotheJHSATdata.Foreachdatacategoryselected,theteamperformedaseparatechisquaredanalysis.Thecategoriesselectedwere:
AccidentsbyOccurrenceCategory VMCOnlyAccidents IMCOnlyAccidents AccidentsbyPilot’sMake/ModelFlightHours
Regardlessofthedatacategoryusedforanalysis,theconclusionwasthesame.Theproportionsoffatalversusnon‐fatalaccidentsintheJHIMDATdatawerenotstatisticallydifferentwhencomparedtotheJHSATdata.
19
AccidentsbyPilot’sFlightHoursTheteamanalyzedaccidentsfromtheJHIMDATdataandtheJHSATdatausingtwodifferenttypesofpilot’sflighthours:rotorcraftflighthoursandmake/modelflighthours.Amajorlimitationwasassociatedwithdrawingconclusionsrelatedtotheproportionofaccidentsascategorizedbypilot’sflighthoursinaspecificflighthourincrement.Thelimitationwasthattheactualflighthourswereunknownforthenon‐accidentU.S.helicopterpilots.Historicalstudiesthatpre‐dateIHSTsuggestedthepercentageofhelicopteraccidentsassociatedwithaspecificincrementofpilot’sflighthourshaveahighpositivecorrelationtothepercentageofthetotalpopulationofhelicopterpilots(accidentpilotsandnon‐accidentpilots)withineachcorrespondingpilot’sflighthourincrement.Ifthesepasthistoricalstudiescontinuetobereliable,thefollowingexampleillustratestheimplications.Itwouldbepossiblethatifthepercentageofaccidentsistwiceashighforpilotswith0‐500hoursrotorcrafthoursascomparedtopilotswith501‐1,000rotorcrafthours,thereasonmaybeattributabletothetotalpopulationofpilotswith0‐500rotorcrafthoursbeingtwiceaslargeasthepopulationofpilotswith501‐1,000rotorcrafthours.Thisconclusionisquitedifferentthanconcludinglowerflighthoursalone(lessflyingexperience)isthecauseofwhypilotswith0‐500rotorcrafthourshavetwiceasmanyaccidentsasthoseinthe501‐1,000hourincrement.Inthecontextofacomparativeanalysissuchasthisreport,thesamelimitationapplies.Theunderlyingreasonaspecificpilot’sflighthourincrementmayshowastatisticaldifferenceinproportionofaccidentsbetweentheJHIMDATdataandtheJHSATdatamaybebecausetheflighthourincrementsforthetotalpopulationofpilotsmayhavechanged.Insummary,anyconclusionsrelatedtoanincreaseordecreaseinthepercentageofrotorcraftaccidentsassociatedwithaspecificpilotflighthourincrementmustincludedeliberationoftheincreasesordecreasesassociatedwiththetotalpopulationofpilotsinthatsameflighthourincrement.RotorcraftFlightHoursTheteamanalyzedtheaggregategroupofaccidentsfromtheJHIMDATdatabyrotorcraftflighthourincrementsagainsttheaggregategroupofJHSATdatabyrotorcraftflighthourincrements.Weusedflighthourincrementsof500flighthoursfrom0upto7,500flighthours(e.g.0‐500hours,501‐1,000hours,1,001‐1,500hours,etc.).Therewereaccidentswherethepilothadgreaterthan7,500rotorcraftflighthours.However,thenumberofpilotspereach500hourincrementabove7,500rotorcraftflighthourswaslow.Thesecasescouldnotbeincludedintheanalysiswithoutsacrificingtheaccuracyofthestatisticaltest.Accordingtothechisquaredanalysis,therewasnotastatisticallysignificantdifferencebetweenthetwodatasetswhencomparingtheaccidentsbyrotorcraftflighthourincrements.AccidentswithPilot’sRotorcraftFlightHours<or=600HoursCaseswherethepilotintheaccidenthadlessthanorequalto600rotorcraftflighthourswereorganizedbyincrementsof50flighthoursfrom0upto600flighthours(e.g.0‐50hours,51‐100hours,101‐150hours,etc).Theteam’sinitialapproachwastoincludeallaccidentswherethepilot’srotorcraftflighthourswerebelow1,000hours.However,thenumberofpilotspereach50hourincrementbetween600and1,000rotorcrafthourswaslow.Thesecasescouldnotbe
20
includedintheanalysiswithoutsacrificingtheaccuracyofthestatisticaltest.Accordingtothechisquaredanalysis,therewasnotastatisticallysignificantdifferencebetweenthetwodatasetswhencomparingaccidentsbyrotorcraftflighthoursforpilotswith600hoursorless.
AccidentsbyPilot’sMake/ModelFlightHoursTheteamanalyzedtheproportionofaccidentscategorizedbypilot’smake/modelflighthoursforboththeJHIMDATandJHSATanalyses.Theanalysisconsistedoforganizingthemake/modelflighthoursbyincrementsof500flighthoursfrom0upto4,500flighthours.Therewereaccidentswherethepilotshadgreaterthan4,500make/modelflighthours.However,thenumberofpilotspereach500hourincrementabove4,500make/modelflighthourswaslow.Thesecasescouldnotbeincludedintheanalysiswithoutsacrificingtheaccuracyofthestatisticaltest.Accordingtothechisquaredanalysis,therewasnotastatisticallysignificantdifferencebetweenthetwodatasetswhencomparingtheaccidentsbymake/modelflighthourincrements.Figure1illustratesthedistributionofpilots’make/modelflighthours.Thefigurealsoincludestheaccidentswherethepilot’smake/modelflighthoursexceeded4,500flighthoursthatwereexcludedfromthechisquaredanalysis.Forpurposesofconsolidatingthedatainthefigure,allcaseswherethepilot’smake/modelflighthoursexceeded10,000flighthourswereincludedinonecategory,showninthetableas“10,001+”.Figure1.DistributionofAccidents‐Pilots'Make/ModelFlightHours
*45JHIMDATaccidentsand51JHSATaccidentsdidnothavepilotmake/modeltime
21
AccidentswithPilot’sMake/ModelFlightHours<or=1,000HoursTheteamanalyzedtheproportionofaccidentsfromboththeJHIMDATandJHSATanalysesforcaseswherethepilotintheaccidenthadlessthanorequalto1,000make/modelflighthours.Theanalysisconsistedoforganizingthemake/modelflighthoursbyincrementsof50flighthoursfrom0upto1,000flighthours.Accordingtothechisquaredanalysis,theJHIMDATdatawasstatisticallydifferentfromtheJHSATdata.Theflighthourincrementswherethenumberofaccidentschangedenoughtobeamajorcontributortothestatisticaldifferencewere:
FlightHourIncrement Change451‐500hours Increasedfrom2.0%to4.3%ofaccidents651‐700hours Increasedfrom1.6%to3.6%ofaccidents801‐850hours Increasedfrom0.7%to2.4%ofaccidents
22
SummaryandConclusionsInthecomparativeanalysisofU.S.helicopteraccidentsfromtheJHIMDATdata(2009‐2011)againsttheJHSATdata(2000,2001,and2006),theteamobservedthateitherstagnancyorregressionwasthemostevidentcharacteristic.Themostfrequentlyciteditemswereconsistentbetweenthedatasetsinmanyofthedifferentanalysisareas.IntheIndustryanalysis,thetopfiveindustriesfromTable1werethesameandappearedinthesamerankorderfromtheJHSATdatatotheJHIMDATdata:1)Personal/Private,2)Instructional/Training,3)AerialApplication,4)EmergencyMedicalServices,and5)Commercial.Thesefiveindustriesaccountedforabout70%ofallU.S.accidentsintheJHIMDATdata,soimplementationmeasuresmusttargetmoreimprovementintheseindustriestostimulatemoresignificantaccidentreductions.Regrettably,thestatisticalanalysisshowednosignificantdifferencebetweentheproportionsofaccidentsinfourofthetopfiveindustries,anindicationofstagnancy.TheonlyIndustrywheretheteamnotedastatisticallysignificantdifferencewasinAerialApplication,whereregressionoccurredasevidentbythesignificantlyhigherproportionofaccidents.TheIndustry/ActivitypairsfromTable3reinforcedtheobservationsabouthowalargevolumeoftheaccidentsaredisproportionatelyrepresentedbyasmallnumberofindustries.ThethreemostfrequentlycitedIndustry/ActivitypairsfromtheJHIMDATanalysisaccountedforover40%oftheaccidentsfrom2009‐2011.AccidentsfromthePersonal/PrivateIndustryengagedinaPersonal/PrivateActivityaccountedforthehighestpercentageofaccidentsobservedintheJHIMDATdataandshowedastatisticallysignificantincreaseintheproportionofaccidentsfromtheJHSATdata.ThesecondandthirdmostfrequentIndustry/ActivitypairingsintheJHIMDATdataweretiedtotheInstructional/TrainingandAerialApplicationIndustrysectors.BothofthesepairingsshowedstagnancyinthattherewasnostatisticallysignificantchangeintheproportionofaccidentsobservedintheJHIMDATdatawhencomparedtotheJHSATdata.AseparatebutstillperplexingproblemintheIndustryandActivitypairsishowfrequentlythePositioning/RTBActivityresultsinaccidentsforboththeEmergencyMedicalServiceandAerialApplicationindustries.Positioning/RTBinvolvesmovingthehelicopterfromonelocationtoanotherandisnottypicallycharacterizedbythesamelevelofcomplexityaseitheroftheseIndustrysectorswouldexperienceinexecutingtherestoftheirtypicalflightprofile.Yet,Positioning/RTBwaspreciselytheActivitythataccountedforthehighestnumberofaccidentsineachofthesetwoIndustrysectors.IntheOccurrenceCategoryanalysisfromTable4,therankorderforfiveoutofthetopsixOccurrenceCategoriesintheJHIMDATdatamatchedtheJHSATdataandprovidedfurtherevidencethattheareasinneedofthestrongestimplementationmeasureshadnotchanged.LossofControlcontinuedtohavethehighestfrequencyofoccurrenceandalsoshowedastatisticallysignificantregression.OneofthepositiveobservationsintheOccurrenceCategoryanalysiswasthattheSystemComponentFailurecategoryhadastatisticallysignificantimprovement.TheSub‐OccurrenceCategoryanalysisfromTable5revealedthiswasattributabletoastatistically
23
significantimprovementinthehelicopterrelatedSystemComponentFailures(versusthosethatwereenginerelated).Unfortunately,therestofthemostfrequentlycitedSub‐OccurrenceCategoriesshowedeitherstagnationorregression.The21%reductioninthenumberofaccidentsthatoccurredinthethreeyearscoveredbytheJHIMDATdata(415accidents)ascomparedtothethreeyearsintheJHSATdata(523accidents)isprogressandshouldnotbedisregardednorminimized.However,thecomparativeanalysisbetweenthetwodatasetsclearlyshowedthattheareasresponsibleforthemajorityofU.S.helicopteraccidentsrequiremoreattentioninordertoachieveamoresignificantreductionintheoverallnumberofaccidents.Akeyelementmissingfromthecomparativeanalysisbetweenthedatasetswasclearanswerstocritical“Why”questions.Forexample,“Whyisstagnancyandregressionsoprevalentintheanalysis?”or“Whyhaveimplementationmeasuresbeenlesseffectiveintheseareas?”TheoriginofthesequestionsisthatwewereuncertainastowhatextentanyofthetoolsandresourcesdevelopedbytheIHSThadeverreachedactualimplementationwithhelicopteroperators.Theambiguityleftopenseveralpossibilities.Perhaps,somedegreeofimplementationoccurredyetdidnotproduceanyimprovement.Unfortunately,thereisalsothepossibilitythatimplementationmayneverhaveoccurredatall.Therewasnoclearevidenceforeitherofthesetwoscenariosoranyothers,sothisreportchosenottospeculatefurther.However,theprevalenceofstagnancyandregressioninthecomparativeanalysismadeitclearthattheimplementationmeasuresforthehighestaccidentproducingIndustrysegmentsshouldbereevaluated.TheUSHSTmusteitherdevelopmorerigorousimplementationmeasuresorformulateamoreeffectiveplanforensuringthatoperatorsareactivelyusingtheimplementationresourcesalreadyavailable.
24
ReferencesU.S.JointHelicopterSafetyAnalysisTeam.(2011,August).TheCompendiumReport:TheU.S.
JHSATBaselineofHelicopterAccidentAnalysis,VolumeI.Retrievedfromhttp://www.ihst.org/portals/54/US_JSHAT_Compendium_Report1.pdf
U.S.JointHelicopterSafetyAnalysisTeam.(2011,July).TheCompendiumReport:TheU.S.JHSATBaselineofHelicopterAccidentAnalysis,VolumeII.Retrievedfromhttp://www.ihst.org/portals/54/US_JSHAT_Compendium_Report2.pdf
Top Related