Critical Success Factors 0407

8/2/2019 Critical Success Factors 0407

1/135

TheCriticalSuccessFactorMethod:EstablishingaFoundationforEnterpriseSecurityManagementAuthorRichardA.CaralliPrincipleContributorsJamesF.StevensBradfordJ.WillkeWilliamR.Wilson

July2004

TECHNICALREPORTCMU/SEI-2004-TR-010ESC-TR-2004-010


2/135


3/135

Pittsburgh,PA15213-3890

TheCriticalSuccessFactorMethod:EstablishingaFoundationforEnterpriseSecurityManagementCMU/SEI-2004-TR-010ESC-TR-2004-010

AuthorRichardA.CaralliPrincipleContributorsJamesF.StevensBradfordJ.WillkeWilliamR.Wilson

July2004

NetworkedSystemsSurvivabilityProgramSurvivableEnterpriseManagementTeam

Unlimiteddistributionsubjecttothecopyright.


4/135

ThisreportwaspreparedfortheSEIJointProgramOfficeHQESC/DIB5EglinStreetHanscomAFB,MA01731-2116TheideasandfindingsinthisreportshouldnotbeconstruedasanofficialDoDposition.Itispublishedintheinterestofscientificandtechnicalinformationexchange.

FORTHECOMMANDER

ChristosScondrasChief

of

Programs,

XPK

ThisworkissponsoredbytheU.S.DepartmentofDefense.TheSoftwareEngineeringInstituteisafederallyfundedresearchanddevelopmentcentersponsoredbytheU.S.DepartmentofDefense.Copyright2004CarnegieMellonUniversity.NOWARRANTYTHISCARNEGIEMELLONUNIVERSITYANDSOFTWAREENGINEERINGINSTITUTEMATERIALISFURNISHEDONAN"AS-IS"BASIS.CARNEGIEMELLONUNIVERSITYMAKESNOWARRANTIESOFANYKIND,EITHEREXPRESSEDORIMPLIED,ASTOANYMATTERINCLUDING,BUTNOTLIMITEDTO,WARRANTYOFFITNESSFORPURPOSEORMERCHANTABILITY,EXCLUSIVITY,ORRESULTSOBTAINEDFROMUSEOFTHEMATERIAL.CARNEGIEMELLONUNIVERSITYDOESNOTMAKEANYWARRANTYOFANYKINDWITHRESPECTTOFREEDOMFROMPATENT,TRADEMARK,ORCOPYRIGHTINFRINGEMENT.Useofanytrademarksinthisreportisnotintendedinanywaytoinfringeontherightsofthetrademarkholder.Internaluse.Permissiontoreproducethisdocumentandtopreparederivativeworksfromthisdocumentforinternaluseisgranted,providedthecopyrightand"NoWarranty"statementsareincludedwithallreproductionsandderivativeworks.Externaluse.RequestsforpermissiontoreproducethisdocumentorpreparederivativeworksofthisdocumentforexternalandcommercialuseshouldbeaddressedtotheSEILicensingAgent.ThisworkwascreatedintheperformanceofFederalGovernmentContractNumberF19628-00-C-0003withCarnegieMel-lonUniversityfortheoperationoftheSoftwareEngineeringInstitute,afederallyfundedresearchanddevelopmentcenter.TheGovernmentoftheUnitedStateshasaroyalty-freegovernment-purposelicensetouse,duplicate,ordisclosethework,inwholeorinpartandinanymanner,andtohaveorpermitotherstodoso,forgovernmentpurposespursuanttothecopy-rightlicenseundertheclauseat252.227-7013.ForinformationaboutpurchasingpapercopiesofSEIreports,pleasevisitthepublicationsportionofourWebsite(http://www.sei.cmu.edu/publications/pubweb.html).


5/135

CMU/SEI-2004-TR-010 i

TableofContents

TotheReader ........................................................................................................ viiAcknowledgements................................................................................................ ixAbstract .................................................................................................................. xi1 Introduction ..................................................................................................... 1

1.1 CriticalSuccessFactors............................................................................ 21.2 EnterpriseSecurityManagement .............................................................. 2

2 Background..................................................................................................... 52.1 LessonsfromOCTAVE ............................................................................. 52.2 ChallengesforSecurityManagement........................................................ 62.3 AddressingChallengeswithCSFs ............................................................ 7

3 HistoryoftheCSFMethod ............................................................................. 93.1 Beginnings ................................................................................................ 93.2

Evolution

of

the

CSF

Method .................................................................. 10

4 ACSFPrimer ................................................................................................. 11

4.1 CSFsDefined.......................................................................................... 114.2 GoalsVersusCSFs................................................................................. 12

4.2.1 RelationshipBetweenGoalsandCSFs....................................... 134.2.2 CardinalityBetweenGoalsandCSFs.......................................... 154.2.3 TheSuperiorityofCSFsOverGoals............................................ 15

4.3 SourcesofCSFs..................................................................................... 164.3.1 IndustryCSFs.............................................................................. 174.3.2 Competitive-PositionorPeerCSFs ............................................. 184.3.3 EnvironmentalCSFs.................................................................... 184.3.4 TemporalCSFs ........................................................................... 194.3.5 Management-PositionCSFs........................................................ 21

4.4 DimensionsofCSFs ............................................................................... 224.4.1 InternalVersusExternal .............................................................. 224.4.2 MonitoringVersusAdapting......................................................... 23


6/135

ii CMU/SEI-2004-TR-010

4.4.3 ImportanceofCSFSourcesandDimensions...............................234.5 HierarchyofCSFs ...................................................................................23

4.5.1 EnterpriseCSFs...........................................................................244.5.2 OperationalUnitCSFs .................................................................254.5.3 RelationshipBetweenHierarchyandSource ...............................264.5.4

Other

Considerations ...................................................................28

5 ApplyingCSFs ...............................................................................................29

5.1 HistoricalApplicationofCSFs..................................................................295.2 GeneralAdvantagesofaCSF-BasedApproach ......................................305.3 UsingCSFsinaSecurityContext............................................................30

5.3.1 EnterpriseSecurityManagement .................................................325.3.2 InformationSecurityRiskAssessmentandManagement.............34

AppendixA CSFMethodDescription ................................................................45AppendixB CaseStudy1:Federal GovernmentAgency ................................91AppendixC CaseStudy2:LargeCountyGovernment ..................................103AppendixD Glossary........................................................................................113References ...........................................................................................................117


7/135

CMU/SEI-2004-TR-010 iii

ListofFigures

Figure1: StrategicPlanninginOrganizations ........................................................ 1Figure2: AlignmentofStrategicPlanandSecurityStrategy .................................. 3Figure3: Goalsvs.CSFs..................................................................................... 13Figure4: RelationshipBetweenGoalsandCSFs ................................................ 15Figure5: ExampleofIndustryCSFsforanAirline................................................ 17Figure6: ExampleofPeerCSFsforanAirline..................................................... 18Figure7: ExampleofEnvironmentalCSFsforanAirline...................................... 19Figure8: ExampleofTemporalCSFsforanAirline.............................................. 21Figure9: ExampleofManagement-PositionCSFsforanAirlineManager........... 22Figure10: ExampleofHierarchyofCSFsinanOrganization ................................ 24Figure11: RelationshipBetweenEnterpriseandOperationalUnitCSFs............... 28Figure12: AffinityAnalysisforDeterminingISRMScope....................................... 36Figure13: AffinityAnalysisforDeterminingCriticalAssets..................................... 38Figure14: AffinityAnalysisforDetermining/ValidatingSecurityRequirements....... 39Figure15: AffinityAnalysisforValidatingEvaluationCriteria.................................. 42Figure16: AffinityAnalysisforDeterminingWhichRiskstoMitigate ...................... 43Figure17: SampleMissionStatement.................................................................... 67Figure18: ExampleofDerivingActivityStatementsfromMission .......................... 68Figure19: ExampleofCSFInterviewNotes .......................................................... 71


8/135

iv CMU/SEI-2004-TR-010

Figure20: ExampleofActivityStatementsDrawnfromCSFInterviewNotes .........71Figure21: AffinityGroupingExampleActivityStatements....................................72Figure22: AffinityGroupingExampleThreeAffinityGroups ................................73Figure23: AffinityGroupingExampleRefinedGroups.........................................73Figure24: ExampleofCSFAffinityGroupingofActivityStatements.......................76Figure25: ExampleofThreeEmergingSupportingThemes ..................................77Figure26: IllustrationofAffinityGroupingofSupportingThemes ...........................80Figure27: IllustrationofDerivingCSFsfromSupportingThemes ..........................82Figure28: ExampleofAffinityAnalysis...................................................................86


9/135

CMU/SEI-2004-TR-010 v

ListofTables

Table1: MatrixofCSFLevelstoCSFTypes ...................................................... 27Table2: CSFInterviewQuestionsProposedbyRockhart .................................. 59Table3: AdditionalInterviewQuestionstoConsider........................................... 61Table4: ExampleofActivityStatementsandSupportingThemes....................... 66Table5: QualitiesofGoodandPoorCSFs .................................................... 83Table6: AgencyCSFs........................................................................................ 93Table7: VulnerabilitiestoAgencyCSFs ............................................................. 98Table8: CountyCSFs ...................................................................................... 106Table9: AffinityAnalysisCSFstoCriticalAssets ............................................111Table10: AffinityAnalysisCSFstoEnterpriseSecurityStrategies................... 112


10/135

vi CMU/SEI-2004-TR-010


11/135

CMU/SEI-2004-TR-010 vii

TotheReader

ThistechnicalreportisbasedontheworkofJohnRockhartandhiscolleaguesattheCenterforInformationSystemsResearch(CISR)attheMassachusettsInstituteofTechnologyintheareaofcriticalsuccessfactorsandinformationsystemsplanning.1 InourresearchattheSoftwareEngineeringInstitute(SEI)intheareasofenterprisesecuritymanagementanden-terpriseresiliency,wefoundbroadapplicabilityofRockhartsconceptsasanimportanttoolindevelopinganddeployinganeffectiveapproachtosecuritymanagement. TheuseofRockhartsconceptsforthispurposeformsthebasisofthistechnicalreport.

Inthis

report,

we

introduce

readers

to

the

critical

success

factors

(CSFs)

concept

and

acorre-

spondingmethodfordevelopingaworkingsetofCSFsthatwedevelopedattheSEI. Moreimportantly,wediscussouruseofCSFsasameansforframingandfocusingthesecuritystrategy,goals,andactivitiesofanorganization. Forbackground,thehistoryandearlyusesofthecriticalsuccessfactormethodinthefieldofinformationsystemsplanningarepre-sented. Withregardtoenterprisesecuritymanagementandenterpriseresiliency,wediscussourrecentapplicationoftheCSFmethodinfieldworkwithcustomersusingtheOperation-allyCriticalThreat,Asset,andVulnerabilityEvaluationSM(OCTAVE)riskassessmentmeth-odology. Thehigh-levelstepswedefinedandappliedtodevelopCSFsforthesecustomersarecodifiedinthisreportforfurtherapplicationandresearch. Finally,wediscussotherwaysinwhichtheCSFmethodcanbeapowerfulguidinganddirectingactivityforthedefinitionandimprovementofenterprisesecuritymanagementprocessesandpracticesinorganizations.Dependingonyourleveloffamiliaritywiththeconceptofcriticalsuccessfactors,thereareseveralwaystomakeeffectiveuseofthematerialpresentedinthisreport. Tofacilitateyouruseofthismaterial,wesuggestthefollowing: IfyouhavenofamiliaritywiththeconceptofcriticalsuccessfactorsortheworkofJohn

Rockhart,youshouldreadeachofthesectionsofthisreportinnumericalsequence. Ifyouarealreadyfamiliarwiththeconceptofcriticalsuccessfactorsandareinterested

inourapplicationofCSFsintheareasofenterprisesecuritymanagementandenterpriseresiliency,youshouldbeginreadingthisreportatChapter5, ApplyingCSFs, andcon-

1 Rockhartsconceptsaredocumentedin APrimeronCriticalSuccessFactors, publishedbytheCenterforInformationSystemsResearchinJune1981[Rockhart81]. Ouruseofthismaterialasthebasisofourresearchhasbeengrantedbypermissionoftheauthor.

SM OperationallyCriticalThreat,Asset,andVulnerabilityEvaluationisaservicemarkofCarnegieMellonUniversity.

OCTAVEisregisteredintheU.S.PatentandTrademarkOfficebyCarnegieMellonUniversity.


12/135

viii CMU/SEI-2004-TR-010

tinuewithAppendicesBandC,whichdescribeourfieldexperienceusingCSFsincus-tomerengagements.

Finally,ifyouhavefamiliaritywithCSFsandareinterestedinobtainingasystematicmethodfordevelopingasetofCSFs,referdirectlytoAppendixA, CSFMethodDe-scription.

Howeveryoudecidetoreadthistechnicalreport,itisourhopethatyouwillseethepotentialbenefitsofderivingandapplyingcriticalsuccessfactorsinyourorganizationandwillrealizeimprovementindevelopinganddeployingyourorganizationalsecuritystrategythroughthissimple,yetpowerfulconcept.


13/135

CMU/SEI-2004-TR-010 ix

Acknowledgements

TheauthorswouldliketothankmembersoftheSurvivableEnterpriseManagementteamoftheNetworkedSystemsSurvivabilityProgramwhohelpedintheproductionofthisreportbyapplyingtheCSFmethodinfieldworkwithcustomersandgraciouslysharingtheirexperi-enceswithus.TheauthorswouldalsoliketothankJuliaAllenofthePractices,Development,andTrainingteamforherreviewofthismaterialandherconsiderablefeedback. Weappreciatehersup-

portandwillingnesstoexploretheseemergingideaswithus.WearealsogratefultoDavidBiberforhisextensiveworkincreatingthegraphicsthatsoappropriatelyillustrateourthoughtsandconceptsandtoPamelaCurtisforhercarefuleditingofthisreport.Wewouldalsoliketothankoursponsorsfortheirsupportofthiswork. Ithasalreadyhadgreatimpactonourcustomersabilitytoimprovetheirsecurityprogramsandinourabilitytotransitionnewtechnologiesintheareaofenterprisesecuritymanagementandenterprisere-siliency.Last,butcertainlynotleast,wewouldliketothankJohnRockhart,whoseworkintheareaofcriticalsuccessfactorsisstillviabletoday. Hisworkimprovedinformationsystemsplanningformanyorganizations,andwehopethatourapplicationofCSFswillhavethesameimpactinthefieldofinformationsecurityandenterprisesecuritymanagement.


14/135

x CMU/SEI-2004-TR-010


15/135

CMU/SEI-2004-TR-010 xi

Abstract

Everyorganizationhasamissionthatdescribeswhyitexists(itspurpose)andwhereitin-tendstogo(itsdirection). Themissionreflectstheorganizationsuniquevaluesandvision.Achievingthemissiontakestheparticipationandskilloftheentireorganization. Thegoalsandobjectivesofeverystaffmembermustbeaimedtowardthemission. However,achievinggoalsandobjectivesisnotenough. Theorganizationmustperformwellinkeyareasonaconsistentbasistoachievethemission. Thesekeyareas uniquetotheorganizationandtheindustryinwhichitcompetes canbedefinedastheorganizationscriticalsuccessfactors.

Thecritical

success

factor

method

is

ameans

for

identifying

these

important

elements

of

suc-

cess. Itwasoriginallydevelopedtoaligninformationtechnologyplanningwiththestrategicdirectionofanorganization. However,inresearchandfieldworkundertakenbymembersoftheSurvivableEnterpriseManagement(SEM)teamattheSoftwareEngineeringInstitute,ithasshownpromiseinhelpingorganizationsguide,direct,andprioritizetheiractivitiesfordevelopingsecuritystrategiesandmanagingsecurityacrosstheirenterprises.Thisreportde-scribesthecriticalsuccessfactormethodandpresentstheSEMteamstheoriesandexperi-enceinapplyingittoenterprisesecuritymanagement.


16/135

xii CMU/SEI-2004-TR-010


17/135

CMU/SEI-2004-TR-010 1

1 Introduction

Anorganization2primarilyexiststoserveitsstakeholders thecustomers,employees,busi-nesspartners,shareholders,andcommunitiesthatbenefitfromtheorganizationsexistenceandgrowth. Theorganizationsmissionembodiesthisfocusbystatingtheorganizations

purpose,vision,andvalues. Stakeholdersarebestservedwhenanorganizationoperatesinamannerthatensuresthemissionisaccomplished.Accomplishingthemissioninalogicalandsystematicwayrequirestheorganizationtode-velopastrategy. Thestrategyencompassesasetofgoalsortargetsthattheorganizationmustachieveinaspecificperiodoftime. Thesegoalsaretransformedintolowerleveltactical

plansandactivitiestobecarriedoutatvariouslevelsthroughouttheorganization. Thisproc-essofstrategicplanningprovidesameansforensuringthattheentireorganizationisfocusedonasharedpurposeandvision.

Figure1: StrategicPlanninginOrganizations2 Itisourintentiontoapplytheterm organization inthisreportuniversallyto for-profit and

non-profit organizations. Whilethebottom-lineobjectivesmaybedifferent,wefindnousefuldistinctionbetweenthesetypesoforganizations bothareinoperationtoaccomplishaspecificmission.


18/135

2 CMU/SEI-2004-TR-010

However,settinggoalsanddevelopingplanstoachievethemisonlyonefactorinaccom-plishingtheorganizationsmission. Theorganizationmustalsoperformwellinafewkeyareasthatareuniquetoitsmissionandtotheindustryinwhichitoperates. Infact,failureto

performwellintheseareasmaybeamajorbarriertoachievinggoals. Thesekeyareascanbedescribedasasetofcriticalsuccessfactors thelimitednumberofareasinwhichsatis-factory

results

will

ensure

competitive

performance

for

the

organization

and

enable

it

to

achieveitsmission[Rockhart79].

1.1 CriticalSuccessFactorsCriticalsuccessfactors(CSFs)definekeyareasofperformancethatareessentialfortheor-ganizationtoaccomplishitsmission. Managersimplicitlyknowandconsiderthesekeyareaswhentheysetgoalsandastheydirectoperationalactivitiesandtasksthatareimportanttoachievinggoals. However,whenthesekeyareasofperformancearemadeexplicit,theypro-videacommonpointofreferencefortheentireorganization. Thus,anyactivityorinitiativethattheorganizationundertakesmustensureconsistentlyhighperformanceinthesekeyar-eas;otherwise,theorganizationmaynotbeabletoachieveitsgoalsandconsequentlymayfailtoaccomplishitsmission.

1.2 EnterpriseSecurityManagementManagingsecurity3acrossanenterpriseisoneofthemanybusinessproblemsthatorganiza-tionsmustsolveinordertoaccomplishtheirmissions. Regardlessofwhatorganizationalassetsaretobesecured informationortechnicalassets,physicalplant,orpersonnel theorganizationmusthaveasecuritystrategythatcanbeimplemented,measured,andrevisedasthebusinessclimateandoperationalenvironmentchange. Inthelongrun,theeffectivenessofthesecuritystrategydependsonhowwellitisalignedwithandsupportstheorganizations

businessdrivers:4mission,businessstrategy,andCSFs.

3 Managingsecuritybroadlyreferstotheprocessofdeveloping,implementing,andmonitoringanorganizationssecuritystrategy,goals,andactivities.

4 Throughoutthisdocumentweusetheterm businessdrivers tocollectivelyrepresenttheorgani-zationsmission,values,andpurpose;itsgoalsandobjectives;anditscriticalsuccessfactors.


19/135

CMU/SEI-2004-TR-010 3

Figure2: AlignmentofStrategicPlanandSecurityStrategy


20/135

4 CMU/SEI-2004-TR-010


21/135

CMU/SEI-2004-TR-010 5

2 Background

TheworkoftheSurvivableEnterpriseManagement(SEM)teamoftheNetworkedSystemsSurvivability(NSS)programattheCarnegieMellonSoftwareEngineeringInstitute(SEI)isfocusedonhelpingorganizationsimprovetheircapabilitiesformanagingsecurityacrosstheirenterprises. Aprimaryobjectiveofthisworkistoestablishstrategicplanningandriskman-agementasessentialcomponentsofasecuritymanagementprogram.Inthissection,wedocumentsomeofthelessonslearnedfromourdevelopmentandfield-workefforts. Inaddition,weintroducetheuseofCSFsasanimportantelementofanor-ganizationsstrategicplanforsecurity.

2.1 LessonsfromOCTAVEOneoftheprimaryfunctionsofexecutive-levelmanagement5istomanageriskacrosstheorganization. Anorganizationssecuritystrategyandgoalsmustbeframedinthecontextofrisktogettheattentionofexecutive-levelmanagement. Onlythoseriskstocriticalassetsthatthreatentheaccomplishmentofthemissionareworthexecutive-levelmanagementsat-tention,andthenonlyiftheorganizationwouldbesignificantlyimpactediftherisksarereal-ized.Arisk-basedapproachtosecuritystrategyandmanagementenablesorganizationstodirecttheirlimitedresourcestotheoperationalareasandcriticalassetsthatmostneedtobepro-tected. Riskstooperationalareasandassetsthatcandirectlyaffecttheorganizationsabilitytoaccomplishitsmissionmustbeidentified,analyzed,andmitigated. Thisperspectiveof focusingonthecriticalfew isafoundationoftheOCTAVEinformationsecurityriskas-sessmentmethodology[Alberts01].InOCTAVE,thisprincipleisputintopracticebycreatinganassessmentteamthatiscom-

posedofpersonnelfromtheorganizationwhounderstandtheorganizationsuniquebusinessdriversandconditions. Implicitly,thesepersonnelarelikelytoconsidertheorganizations CarnegieMellonisregisteredwiththeU.S.PatentandTrademarkOffice.5 Inthisreport,termexecutive-levelmanagementisintendedtorefertothosepersonnelinC-level

(e.g.,CEO)positions,aswellastheirfirst-levelseniormanagers(vice-presidents,executivedirec-tors,etc.). Thesepersonnelareinvolvedintheorganizationsstrategicplanningprocessandareresponsibleforsettingthedirectionandcoursefortheorganization.


22/135

6 CMU/SEI-2004-TR-010

missionwhentheydecidewhichoperationalareasandassetstoincludeintheriskassessmentactivity.Identifyingandfocusingonthemostimportantoperationalareasandassetsisperhapsthemostimportantactivitythatanorganizationperformswhendeployingarisk-basedapproachtosecurity. However,aswehavelearnedinourfieldworkwiththeOCTAVEmethod,thiscanbeadifficulttaskinalarge,complexorganizationparticularlybecausetheremaybenu-merousoperationalareasfromwhichtochoose,eachwithitsownsetofimportantassets. Ananalysisteammustapplytheirjudgmentinselectingtherightareasandassets,andmusten-surethattheirselectionalignswiththebusinessdriversoftheorganization. Failuretoselect(andvalidate)therightoperationalareasandassetscansignificantlydiminishthevalueofarisk-basedapproachtosecurity.

2.2 ChallengesforSecurityManagementInthepastthreeyears,ourresearch,fieldwork,andclassroominteractionhasprovidedmuchdataregardingthechallengesandbarriersthatorganizationsfaceinmakingthetransitionfromvulnerability-based6torisk-basedapproachestosecuritymanagement. Overall,wehaveobservedthatmanyorganizationsunderstandclearlythatsuccessdependsongainingthesponsorshipofexecutive-levelmanagementandaligningsecuritygoalswiththemission,goals,andobjectivesoftheorganization. Inthisway,securitygoalsbecomeanenableroftheorganizationsmissionorstrategy,ratherthanaburdenorexpense. However,ourexperi-encesuggeststhatmanyorganizationsareill-equippedtodefinetheirsecuritygoals,letalonetomakeanexplicitconnectionbetweentheirsecuritygoalsandthestrategicdrivers7oftheorganization.Thisisnotunlikeasimilarchallengethathasbeenfacedbyinformationtechnology(IT)de-

partmentsinorganizations. Theacceptanceofthepositionofchiefinformationofficer(CIO)asalegitimateexecutive-levelpartnertothechiefexecutiveofficer(CEO)andchieffinancialofficer(CFO)hasbeenamorerecentaccomplishmentinmanyorganizations. LegitimizingthispositioncausestheITdepartmenttobecomeastrategicpartneroftheorganization,help-ingitachieveitsmissionmoreefficientlyandeffectively. Manywell-knownorganizationshaveindeedproventheirabilitytobecompetitive,togrow,andtoaccomplishtheirmissionsthroughinnovativeandstrategicusesoftechnology.

6 Wedescribea vulnerability-based approachtosecurityasoneinwhichtheprimaryfocusistoreacttovulnerabilities(suchasvirusesorintrusions)astheyareidentified,ratherthantotakeaproactive,strategy-drivenapproachtosecurity. Vulnerabilitymanagementisanimportantpartofmanagingsecuritybutrarelyissufficientaloneforsecuringalargeorganizationorenterprise.

7 Inthisreport,theterm strategicdrivers isusedtorefertotheimportantcomponentsofanor-ganizationsstrategicplan:mission,objectives,goals,andcriticalsuccessfactors. Thesedriversmaysometimesbereferredtoas businessdrivers or organizationaldrivers.


23/135

CMU/SEI-2004-TR-010 7

Inthesameway,anorganizationssecuritystrategymustalignwithandenableitsorganiza-tionalstrategy. But,withtheincreasingdependenceoftheorganizationsmissiononinfor-mationtechnology,securitystrategymustalsoensurethattheorganizationisresilientagainstattacks,particularlyontechnology,thatcoulddisablethemission.

Ourconclusionisthatastrongpartnershipislackingbetweenexecutive-levelmanagementandthepartsoftheorganizationresponsibleforsettingandimplementingsecuritystrategy.Toassistourcustomerswiththischallenge,webegantosearchforwaysthatcouldaidinmakingthisconnectionmoreexplicit.

2.3 AddressingChallengeswithCSFsOneofthewaysinwhichITdepartmentshaveaddressedthesechallenges(asearlyasthe1970s)isbyinvolvingtheorganizationatlargeintheirstrategicplanningprocess. This

process knownby

many

names,

such

as

business

systems

planning explicitly

takes

into

considerationtheorganizationskeybusinessprocessesanddatatodeterminethetechnologyneedsoftheorganization. Tofurtherdeterminepriority,theseeffortsalsofrequentlyincludeadirection-settingactivitysuchasthedevelopmentofCSFs. Iftheorganizationsaccom-

plishmentofthemissionistightlylinkedtoitsperformanceinafewkeyareasandthetech-nologyplanisbasedonenablinghighperformanceinthesesameareas,theplancanenablethemission.WedrewuponthebroadexperienceoftheSEMteamtoaddresssimilarchallengesforsecu-ritymanagement. AtleastoneSEMteammemberhadpreviouslyusedCSFsinthedevel-opmentofaninformationtechnologyplan. OtherteammemberswerealsofamiliarwithCSFs,andthuswebegantoexploretheCSFmethodasapossiblewaytohelpourcustomersimprovethefocusoftheirsecurityefforts. Webeganourinvestigationofthemethodspecifi-callyinresponsetotheincreasingnumberofquestionsandconcernsofcustomersintheirattempttodevelopascopefortheirriskassessmentactivities selectingtherightoperationalareasandcriticalassetstofocuson. Inourfieldwork,wealsoobservedthevalueofthemethodforsecuritymanagementandstrategyandgoaldevelopment.


24/135

8 CMU/SEI-2004-TR-010


25/135

CMU/SEI-2004-TR-010 9

3 HistoryoftheCSFMethod

TheconceptofidentifyingandapplyingCSFstobusinessproblemsisnotarevolutionarynewfieldofwork. Itdatesbacktotheoriginalconceptof successfactors putforthinman-agementliteraturebyD.RonaldDanielinthe1960s.8 However,theCSFconceptsandap-

proacharestillverypowerfultodayandareapplicabletomanyofthechallengesbeingpre-sentedintheinformationtechnologyandsecurityfields.

3.1 BeginningsInthelate1970sandearly1980s,organizationsfoundthemselvesinthemidstofaninforma-tionrevolution. Thegrowthofinformationsystemsinorganizationsresultedintheproduc-tionofsignificantamountsofinformationforanalysisanddecisionmaking. Theadventofthepersonalcomputerandtheevolutionofthefieldofinformation systems toinformation technology wereindicatorsthattheinformationexplosionwouldcontinue.JohnF.Rockhart,ofMITsSloanSchoolofManagement,recognizedthechallengethattheonslaughtofinformationpresentedtoseniorexecutives. Inspiteoftheavailabilityofmoreinformation,researchshowedthatseniorexecutivesstilllackedtheinformationessentialtomakethekindsofdecisionsnecessarytomanagetheenterprise[Dobbins98]. Asaresult,Rockhartsteamconcentratedondevelopinganapproachtohelpexecutivesclearlyidentifyanddefinetheirinformationneeds.RockhartsteamexpandedontheworkofDanieltodeveloptheCSFapproach. Danielsug-gestedthat,tobeeffectiveinavoidinginformationoverload,anorganizationsinformationsystemsmustfocusonfactorsthatdetermineorganizationalsuccess[Rockhart79]. Forex-ample,intheautomotiveindustry,Rockhartsuggestedthatstyling,anefficientdealerorgani-zation,andtightcontrolofmanufacturingcostsareimportantsuccessfactors[Rockhart79].Usingsuccessfactorsasafilter,managementcouldthenidentifytheinformationthatwasmostimportanttomakingcriticalenterprisedecisions. Accordingly,theunderlyingpremiseisthatdecisionsmadeinthismannershouldbemoreeffectivebecausetheyarebasedondatathatisspecificallylinkedtotheorganizationssuccessfactors.

8 Danielsconceptsaredescribedin ManagementInformationCrisis, HarvardBusinessReview,September-October1961.


26/135

10 CMU/SEI-2004-TR-010

In1981,Rockhartcodifiedanapproachthatembodiedtheprinciplesofsuccessfactorsasawaytosystematicallyidentifytheinformationneedsofexecutives. Thiswork,presentedin APrimeronCriticalSuccessFactors, detailedthestepsnecessarytocollectandanalyzedataforthecreationofasetoforganizationalCSFs[Rockhart81]. ThisdocumentiswidelyconsideredtobetheearliestdescriptionoftheCSFapproach. Ourinterpretationandapplica-tion

of

Rockharts

approach,

as

documented

in

this

report,

is

largely

based

on

this

description.

3.2 EvolutionoftheCSFMethodMostoftheworkinsuccessfactorsperformedbyRockhartandDanielwasfocusedonrefin-ingtheinformationneedsofexecutives. However,asalogicaloutgrowthofthiswork,Rockharthintedattheusefulnessofthemethodasacomponentofstrategicplanningforin-formationsystemsortechnology[Rockhart81]. TheCSFmethodhasfounditswayintomanyformalizedinformationorbusinesssystemsandtechnologyplanningmethodologiesthatarestillbeingusedtoday.TheCSFmethodandtheanalysisofCSFshavebeenusedinmanywaysoutsideofthein-formationtechnologyplanningarena. IntheirresearchontheuseofCSFsinfederalgov-ernmentprogrammanagement,JamesDobbinsandRichardDonnelly[Dobbins98]identifyusesofCSFsto identifythekeyconcernsofseniormanagement assistinthedevelopmentofstrategicplans identifykeyfocusareasineachstageofaprojectlifecycleandthemajorcausesofpro-

jectfailure evaluatethereliabilityofaninformationsystem identifybusinessthreatsandopportunities measuretheproductivityofpeopleWhilethisisnotanexhaustivelistofthewaysinwhichRockhartsoriginalworkhasbeenapplied,itsuggeststhebroadapplicabilityofthemethod. ItspeakstotheuseofCSFsasawayfororganizationstofocusandvalidatemanyoftheimportantactivitiestheyperformtoaccomplishtheirmissions.


27/135

CMU/SEI-2004-TR-010 11

4 ACSFPrimer9

CSFsareanexplicitrepresentationofthekeyperformanceareasofanorganization. Inthiscontext,CSFsdefinethosesustainingactivitiesthatanorganizationmustperformwellovertimetoaccomplishitsmission. Theyarefoundateverylevelofmanagement,fromexecu-tivetolinemanagement. EachorganizationalsohasasetofCSFsthatitinheritsfromthe

particularindustryinwhichitoperates.ToapplytheCSFmethodandtouseCSFsasananalysistool,itisimportanttounderstandhowtheyrelatetotheorganizationsstrategicdriversandcompetitiveenvironment. ThissectionprovidesafoundationforunderstandingCSFsanddefinestheseimportantrelation-ships.

4.1 CSFsDefinedTheterm criticalsuccessfactor hasbeenadaptedformanydifferentuses. Familiaritywiththetermisoftenpresentedinthecontextofaprojectoraninitiative(i.e.,theCSFsfortheimplementationofanERPsystemorthedeploymentofadiversityprogram). Inthiscontext,CSFsdescribetheunderlyingorguidingprinciplesofaneffortthatmustberegardedtoen-surethatitissuccessful.AslightdistinctionmustbemadewhenconsideringCSFsasastrategicdriverattheorgan-izationalorenterpriselevel(asisdoneinthisreport). Inthiscontext,CSFsaremorethan

justguidingprinciples;instead,theyareconsideredtobeanimportantcomponentofastrate-gicplanthatmustbeachievedinadditiontotheorganizationsgoalsandobjectives. Whilethisdistinctionissubtle,itisintendedtopointoutthatanorganizationsCSFsarenotjustto

be keptinmind ;theirsuccessfulexecutionmustdrivetheorganizationtowardaccomplish-ingitsmission.ManydefinitionsofaCSFatthestrategicplanninglevelhavealreadybeenprovidedinthisreport. InhisseminalworkonCSFs,Rockhartprovidesausefulsummaryofsimilarbutdis-tinctdefinitions[Rockhart81]: keyareasofactivityinwhichfavorableresultsareabsolutelynecessarytoreachgoals9 ThissectionreliesheavilyonthedescriptionofCSFsasdocumentedintheoriginalprimerby

JohnRockhartandChristineBullen[Rockhart81]. TheirworkisstillwidelyrecognizedastheinitialdefinitionofCSFsandtheCSFapproach.


28/135

12 CMU/SEI-2004-TR-010

keyareaswherethingsmustgorightforthebusinesstoflourish factors thatare critical tothe success oftheorganization keyareasofactivitiesthatshouldreceiveconstantandcarefulattentionfrommanage-

ment arelativelysmallnumberoftrulyimportantmattersonwhichamanagershouldfocusattentionThefactthatCSFscanbedefinedinsomanydifferentwaysspeakstotheirelusivenature.ManagersgenerallyrecognizetheirCSFs(andtheorganizations)whentheyseeorhearthem,butmaybeunabletoclearlyandconciselyarticulatethemorappreciatetheirimpor-tance. Infact,mostmanagersareawareofthevariablestheymustmanagetobesuccessful,yetonlywhenproblemsariseandrootcausesareidentifiedarethesevariablesmadeexplicit.Forexample,supposeanorganizationfindsanalarmingnumberofduplicatepaymentstovendors. Theymightconcludethatthisproblemisrelatedtopoorstafftrainingorhighlevelsofstaffturnover.Asaresult,theeffectivemanagementofhumanresources(attracting,train-ing,retaining)mightbeidentifiedasanimportantfactorthatcanimpedetheachievementoftheirstrategicgoals. Intheprocess,theyhaveexplicitlydefinedaCSFfortheorganization.CSFsarepowerfulbecausetheymakeexplicitthosethingsthatamanagerintuitively,repeat-edly,andevenperhapsaccidentallyknowsanddoes(orshoulddo)tostaycompetitive.However,whenmadeexplicit,aCSFcantaptheintuitionofagoodmanagerandmakeitavailabletoguideanddirecttheorganizationtowardaccomplishingitsmission.

4.2 GoalsVersusCSFsIntraditionalstrategicplanningandmanagement,thedefinitionofagoaloranobjectiveisfairlywellknown;however,definingaCSFismuchlessclear[Rockhart81]. Thus,CSFsareoftenconfusedwithorganizationalgoals. Forthepurposeofthisreport,wedefineorgan-izationalgoalsastargetsthatareestablishedtoachievetheorganizationsmission. Theyareveryspecific10astowhatmustbeachieved,whenitistobeachieved,andbywhom. Effec-tivegoalshaveaquantitativeelementthatismeasurabletodetermineifthegoalhasbeenachieved. Goalscanbedecomposedintooperationalactivitiestobeperformedthroughouttheorganization.

10 GoalsshouldbeS.M.A.R.T. specific,measurable,achievable,realistic,andtangible tobeef-fective.Goalsthatdonothavethislevelofspecificitycaneasilybecomeconfusedwithcriticalsuccessfactors. MoreinformationabouttheS.M.A.R.TapproachtogoalsettingcanbefoundinAttitudeisEverything!byPaulJ.Meyer[Meyer04]oronlineathttp://www.topachievement.com/smart.html.


29/135

CMU/SEI-2004-TR-010 13

Figure3: Goalsvs.CSFsGoalsandCSFsgohand-in-hand. Bothareneededtoaccomplishtheorganizationsmission,andneithercanbeignoredwithoutaffectingtheother. Becausetheyarebothintegralpartsofanorganizationsstrategicplan,theirrelationshipmustbeconsidered. Forexample,apersonmighthaveagoaloflosing10poundsbytheendoftheyear. Toachievethisgoal,thepersonwouldhavetobemindfulofafewkeyfactors improvinghisorherdietandnutrition,exer-cisingregularly,andavoidingtemptingsocialgatherings. Carefulattentiontothesekeyfac-torswillenablethepersontoachievethegoaloflosing10pounds;conversely,inattentiontothesefactorswillinhibitachievementofthegoal.

4.2.1RelationshipBetweenGoalsandCSFsThestrongrelationshipbetweengoalsandCSFsresultsfromthefactthatmanagersaretheoriginofbothgoalsandCSFs. Whenmanagerssetgoals,theyalsoimplicitlyconsiderwhattheyneedtodotobesuccessfulatachievingthegoals. Thus,itislikelythatmanagerscon-sciouslyconsidertheirCSFsduringgoalsettingandconsequentlycreatethebondbetween


30/135

14 CMU/SEI-2004-TR-010

goalsandCSFsthatisneededtocontributetoaccomplishingtheorganizationsmission. Inthisway,theinfluenceofCSFsongoalachievementismadeexplicit,eveniftheactualCSFsarenot.OrganizationsthathavebeensuccessfulatachievingtheirgoalshavealsolikelyachievedtheirCSFs,albeitinalessobservableway. Thus,goalssometimesresembleCSFs

becausetheyembodytheimportanceofakeyperformancearea.UsuallyagoalisimmediatelydiscerniblefromaCSFbecauseofitsspecificity. ACSFfortheorganizationmaybemoregeneralandislikelytoberelatedtomorethanonegoal. Con-siderthefollowinggoalsforalargemanufacturingcompany: IncreasesalesinourNortheastdivisionby10%by2ndquarter,2004. Decreasetravelexpensesby5%inthenext30days. Expandproductlinetoincludewidgetsandgadgets. IncreaseexpansionbyopeningatleasttworetailstoresinatleasttwoEuropeanmarkets

by3rdquarter2006.Thefirstgoalmightbecommonlyfoundinmanycommercialorganizations:toachievea10%increaseinsalesinadivisionalunit. Toachievethisgoal,themanufacturingcompanyisstatinganimplicitdependenceontheorganizationsabilitytoperformwellinafewkeyareas. Whilethegoalissimple,itreflectsmanykeyunderlyingassumptionsorconditions.Implicitly,thisgoalstatesthat Thegrowthofthecompanyisdependentontheorganizationscapabilityforincreasing

sales. Salesstaffmustbeempoweredandenabledtomeetthechallengeofattaininganincrease

of10%. Thecompanymustactquicklybecauseitneedstoretainandgrowitsmarketshareinthe

Northeastasothercompetitorsrampup. TheNortheastdivisionisanimportantareainwhichsalesexpansionbringsthecompany

acompetitiveadvantage.TheseassumptionsorconditionsembodyCSFsthataredirectlyrelatedtothepotentialsuc-cessinachievingthegoal. Forexample,considerthefollowingdependenciesbetweenthegoal,underlyingassumptionsandconditions,andCSFs:


31/135

CMU/SEI-2004-TR-010 15

Figure4: RelationshipBetweenGoalsandCSFsTheimportanceoftheCSFsinhelpingthemanufacturingcompanyachieveitsgoalscannot

beoverstated. Inthisexample,atleastoneoftheCSFs attract,train,andretaincompetentsalesstaff isvitallyimportantifthecompanywantstoachievethegoalofattaininga10%increaseinsales. Ifthecompanyfailstoconsistentlyretainqualifiedsalesstaff,thegoalcan-notbeachieved,andinthelongrun,themanufacturingcompanysmissionmaybeinjeop-ardy.

4.2.2Cardinality11BetweenGoalsandCSFsAsillustratedabove,anorganizationalgoalmayberelatedtomorethanoneCSFtobeachieved. Conversely,aCSFmayinfluenceoraffecttheachievementofseveraldifferentgoals. Thepotentialmany-to-manyrelationshipbetweengoalsandCSFsisindicativeoftheirinterdependentnatureandtheimportanceofCSFsinhelpingtheorganizationaccom-

plishitsmission.

4.2.3 TheSuperiorityofCSFsOverGoalsGoalsalonecanbeanunreliablepredictorofanorganizationsabilitytosuccessfullyaccom-

plishitsmission.Thisisbecausegoal-settinginmanyorganizationsisatbestasubjectiveexerciseandoftenisstronglyinfluencedbyorderivedfromaperformancemanagementsys-temratherthanastrategicplanningexercise. Often,goalsaresetwithaneyetotheir11 Cardinalityreferstotheextentoftherelationshipbetweentwoentities. Ausefuldefinitioninthe

contextofCSFsis abusinessrulespecifyinghowmanytimesanentitycanberelatedtoanotherentityinagivenrelationship. (Thisdefinitioncanbefoundathttp://www.vertaasis.com.)


32/135

16 CMU/SEI-2004-TR-010

achievabilityratherthanhowtheycontributetoaccomplishingthemission. Forexample,anorganizationmayrealizethatitisfailingtoaccomplishitsmissioneventhoughithassuc-cessfullyachieveditsgoals. Thiscanoccurbecausethegoalshavenotbeenalignedwiththeorganizationsstrategicplan;thustheirachievementdoesnotpropeltheorganizationfor-ward.Ontheotherhand,CSFsarelesslikelytobebiasedtowardachievement.WhileCSFsarederivedfromandreflecttheconsiderationsofmanagement,theyarealsoinheritedbytheorganizationfromtheindustryinwhichitoperates,itspositionrelativetopeerorganizations,andtheeffectsofthecurrentoperatingclimateandenvironment. Asaresult,eventhoughanorganizationmaynotachieveitsgoals,achievingCSFsmaystillgettheorganizationclosertoaccomplishingthemission. OrganizationsthathaveachievedtheirgoalsbutfailedattheirmissionsmayhaveignoredtheachievementoftheirCSFs.Theconnectionbetweenanorganizationsoperatingenvironmentand CSFsmakethemcol-lectivelymorereliableasapredictoroftheorganizationscapabilitiesforaccomplishingthemission. Tofurtherdevelopthisassertion,itisusefultoexplorethevarioussourcesofCSFsinmoredetail.

4.3 SourcesofCSFsCSFsaregenerallydescribedwithinthesphereofinfluenceofaparticularmanager. Buttherearemanylevelsofmanagementinatypicalorganization,eachofwhichmayhavevastlydifferentoperatingenvironments. Forexample,executive-levelmanagersmaybefo-cusedontheexternalenvironmentinwhichtheirorganizationslive,compete,andthrive. Incontrast,line-levelmanagersmaybeconcernedwiththeoperationaldetailsoftheorganiza-tionandthereforearefocusedonwhattheyneedtodotoachievetheirinternal,operationalgoals. Becauseofthesedifferentoperationaldomains,theCSFsfortheorganizationwillcomefrommanydifferentsources. Allareimportantfortheorganizationasawholetoac-complishitsmission,regardlessoftheirsource.RockhartdefinedfivespecificsourcesortypesofCSFs12fortheorganizationasfollows:[Rockhart81] theindustryinwhichtheorganizationcompetesorexists anunderstandingoftheorganizationspeers thegeneralbusinessclimateororganizationalenvironment12 InourapplicationoftheCSFmethodtosecurityactivities,wedidnotconcernourselvesspecifi-

callywithensuringthatCSFswereidentifiedineachofRockhartscategories. However,consid-erationofeachofthesecategoriesmakesasetofCSFsmorerobustandrepresentativeofallofthevariousoperatingdomainsofanorganization.


33/135

CMU/SEI-2004-TR-010 17

problems,barriers,orchallengestotheorganization layersofmanagementToprovideanaccuratepictureofanorganizationsoverallkeyperformanceareas,itisim-

portanttoidentifyCSFsfromeachofthesesources. However,aswefoundinouruseoftheCSF

method,

deriving

CSFs

at

the

highest

levels

of

the

organization

tends

to

bring

an

accept-

ablemixofCSFsfrommanyofthesesources,solongasabroadcrosssectionofmanage-mentisrepresentedintheprocess.EachsourceofCSFanditsimportancetounderstandingtheorganizationskeyperformanceareasisdiscussedinmoredetailinthefollowingsections.

4.3.1IndustryCSFsEveryorganizationinheritsaparticularsetofoperatingconditionsandchallengesthatareinherenttotheindustry(orsegmentoftheindustry)inwhichitchosetodobusiness. ThisresultsinauniquesetofCSFsthatorganizationsinaparticularindustrymustachievetomaintainorincreasetheircompetitivepositions,achievetheirgoals,andaccomplishtheirmissions. Forexample,consideranorganizationintheairlineindustry. Asamemberofthisindustry,theorganizationinheritsCSFssuchas deliveron-timeservice or moveawayfromthehub-and-spokesystem. FailuretoachievetheseCSFsmayrendertheorganizationunabletostaycompetitiveinitsindustryandmayultimatelyresultinitsexit.

Figure5: ExampleofIndustryCSFsforanAirlineIndustryCSFsdonotnecessarilyapplyonlytoacommercialorprofit-orientedmission. Inreality,theconceptofindustryCSFscanapplytoorganizationsthathaveacommercial,edu-cational,public-service,ornon-profitorientation. Thustheterm industry inthiscontext


34/135

18 CMU/SEI-2004-TR-010

describesanorganizationwhosepurpose,vision,andmissionistypicallysimilartothoseofitspeers.

4.3.2Competitive-PositionorPeerCSFsPeer-groupCSFsareafurtherdelineationofindustry-basedCSFs. TheydefinethoseCSFsthatarespecifictotheorganizationsuniquepositionrelativetotheirpeergroupintheindus-tryinwhichtheyoperateorcompete. Forexample,anorganizationmaybealeaderoralag-gardinaparticularindustry. Iftheyarealeader,theymayhaveCSFsthatareaimedaten-suringtheymaintainorincreasetheirmarketshareagainstotherorganizationsintheindustry.Ontheotherhand,ifconsideredalaggard,theorganizationmayhavespecificCSFsaimedatclosingthegapandimprovingtheircompetitivepositionrelativetootherorganizationsintheirindustry. Inthecaseoftheairline,anexampleofapeer-groupCSFmaybeto reducecostperpassengermile or increasecodesharepartnerships. TheseCSFsmaybeneces-saryforthecompanytoincreasemarketshareinnewgeographicalareasandtomaintainorincrease

their

competitive

positions.

Figure6: ExampleofPeerCSFsforanAirline

4.3.3EnvironmentalCSFsTobesuccessful,anorganizationmustbemindfulofthemacroenvironmentinwhichitop-erates. Aclosedorganization onethatdoesnotfullyinteractwithitsexternalenviron-ment cannotsurviveinthelongterm. Asaresult,anorganizationmustacknowledgethe


35/135

CMU/SEI-2004-TR-010 19

environmentalfactorsthatcanaffectitsabilitytoaccomplishitsmission. EnvironmentalCSFsreflecttheenvironmentalfactorsoverwhichtheorganizationhasverylittlecontrolorabilitytoactivelymanage. Bymakingthesefactorsexplicit,theorganizationcanatleastbemindfulofthemandactivelymonitortheirperformancerelativetothem.

EnvironmentalCSFsdescribesuchconditionsascurrentsocio-politicalissues,theindustrysregulatoryenvironment,andfactorssuchasseasonality. Forexample,theairlineindustryhas

beendramaticallyaffectedbyterroristactivities,whichhaveforcedchangesinairportopera-tionsandschedulingandhavebroughtaboutnewregulationswithwhichairlinesmustcom-

ply. Unfortunately,airlineshaveverylittlecontroloverthisproblem.

Figure7: ExampleofEnvironmentalCSFsforanAirline

4.3.4TemporalCSFsCSFs

are

tied

to

the

long-term

planning

horizon

of

an

organization.

Over

the

strategic

plan-

ningperiodtheorganizationsCSFsmayremainfairlyconstant,adjustedonlywhentheor-ganizationmakesmajorchanges,suchaschangingitsmissionortheindustryinwhichitcompetes.However,atonetimeoranother,everyorganizationencounterstemporarycondi-tionsorsituationsthatmustbemanagedforaspecificperiodoftime,whilecontinuingtomaintainitsperformanceinallotherareas. Thesetemporaryconditionsorsituationscanre-sultintemporalCSFs areasinwhichtheorganizationmusttemporarilyperformsatisfacto-


36/135

20 CMU/SEI-2004-TR-010

rilyinordertoensurethatitsabilitytoaccomplishitsmissionisnotimpeded. Forexample,thefollowingconditionscancreatetemporalCSFs: threatsthathavebeenidentifiedthroughSWOT13analysis temporaryoperatingconditions,suchashighinventorylevelsthatmustbereduced extremechangesintheorganizationsindustry,suchastheeffectofthe9-11terroristat-

tacksontheairlineandtravelindustries barrierstoentrytoanewmarketoranewindustrythatarisewhentheorganizationtakes

onanewstrategicdirection temporaryenvironmentalfactors,suchaswar,extremeweather,lossofkeyemployees processorproductionproblemsthatcausetemporarychangesintheorganizationsability

toproduceitsprimaryproductsorservices lawsuitsorlegalactionsbroughtagainsttheorganizationthatmustbemanagedasa

courseofbusinessuntilresolvedKeepinmindthatatemporalCSFmaybeanindicationofapermanentchangeintheorgani-zationsindustry,operatingenvironment,orcompetitivepositionandasaresultmaybeadoptedasalong-termorganizationalCSFbecauseofitsstrategicimportance.

13 SWOTanalysisisacommonlyusedstrategicplanningtechnique. Itidentifiestheorganizationsstrengths,weaknesses,opportunities,andthreatsthatshouldbeconsideredindevelopingastrate-gicplan.


37/135

CMU/SEI-2004-TR-010 21

Figure8: ExampleofTemporalCSFsforanAirline

4.3.5Management-PositionCSFsEverylayerofmanagementhasadifferentperspectiveandfocusintheorganization. Thisdivisionoflaborensuresthatbothtacticalandstrategicactionsaretakentoaccomplishtheorganizationsmission. Managershavedifferentfocusesandprioritiesdependingonthelayerofmanagementinwhichtheyoperate. ThistranslatesintoasetofCSFsthatreflectthetypeofresponsibilitiesrequiredbythemanagerspositionintheorganization. Infact,theCSFsthatareinherenttothelevelofmanagementmaybeuniversalacrossdifferentorganiza-tionsinthesameindustry. Forexample,executive-levelmanagersmayhaveCSFsthatfocusonriskmanagement,whereasoperationalunitmanagersmayhaveCSFsthataddressproduc-tioncontrolorcostcontrol.


38/135

22 CMU/SEI-2004-TR-010

Figure9: ExampleofManagement-PositionCSFsforanAirlineManager

4.4 DimensionsofCSFsInhisinitialwork,RockhartalsodescribedvariousdimensionsofCSFsthatareusefulforunderstandingaparticularmanagersviewoftheworld[Rockhart81]. CSFscanbecatego-rizedbythesedimensionstofurtherclarifythecurrentfocusoftheorganizationandhowitis

positionedamongitspeers.ThedimensionsofCSFsasdescribedbyRockhartare internal external monitoring adapting4.4.1InternalVersusExternalInternalCSFsarethoseCSFsthatarewithinthespanofcontrolforaparticularmanager. Incontrast,externalCSFsarethoseoverwhichamanagerhasverylittlecontrol. Forexample,


39/135

CMU/SEI-2004-TR-010 23

intheairlineindustryexample,aninternalCSFcouldbe managinggroundoperations, whileanexternalCSFmaybe fuelcosts. CategorizingaCSFaseitherinternalorexternalisimportantbecauseitcanprovidebetterinsightformanagersinsettinggoals. Forexample,amanagercansetveryspecific,achiev-ablegoalsthatcomplementtheachievementofinternalCSFsbecausethemanagerhascon-troloverthem. However,ifamanagerhasanexternalCSF,heorshemustsetgoalsthataimtoachievetheCSFandminimizeanyimpactonoperationsthatmayresultbecausetheCSFisnotinhisorherdirectcontrol.

4.4.2MonitoringVersusAdaptingMonitoringCSFsemphasizethecontinuedscrutinyofexistingsituations[Rockhart81]. Be-causemonitoringtheorganizationshealthisaprimaryfunctionofmanagement,almostallmanagershavesometypeofmonitoringCSF. Infact,inourworkwithCSFs,wehavefoundthatmanyenterpriseCSFs(thosethatapplytotheentireorganization)arefocusedonmoni-toringtheorganizationsperformanceinafewkeyareas,suchascompliancewithregula-tions. Conversely,adaptingCSFsarefocusedonimprovingandgrowingtheorganization.WehavealsofoundthatmanyenterpriseCSFsareadaptingCSFsbecausetheystatetheor-ganizationsdesiretoimprovetheircompetitivepositionortomakeamajorchangeintheirmission. Inthesecases,thedistinctionbetweenagoalandaCSFislessclear whatappearstobeagoaloftheorganizationisactuallyanadaptingCSF.

4.4.3ImportanceofCSFSourcesandDimensionsThesourceanddimensionofaCSFprovidesadditionalinformationforunderstandingtheimportanceofaCSFanditscontributiontotheaccomplishmentoftheorganizationsmis-sion. Tobeeffective,managersmustconsiderandmonitorawiderangeofactivities,events,andconditionsthatoccurthroughouttheorganizationandintheexternalenvironmentinwhichtheorganizationoperates. GatheringCSFsthatincorporateandreflectvariousCSFsourcesanddimensionsprovidesaneffectivedelineationofamanagersfieldofvision arepresentationofthedepthandbreadthofthemanagersresponsibilities.

4.5 HierarchyofCSFsAsexplainedpreviously,CSFsexistthroughoutalllevelsoftheorganizationandcancomefrommanysources. Aswithstrategicplanningandgoalsetting,CSFsathigherlevelsoftheorganizationarerelatedto(ordependenton)thoseatlowerlevelsintheorganization. HigherlevelCSFscannotgenerallybeachievedunlesslowerlevelCSFsareachievedaswell.


40/135

24 CMU/SEI-2004-TR-010

HigherlevelCSFsinfluencelowerlevelCSFs. Infact,iflowerlevelCSFsdiffersignifi-cantlyfromhigherlevelCSFs,theorganizationmustconsiderwhetherthereisproperalign-mentbetweentheactivitiesoflowerlevelmanagementandthestrategicdirectionoftheor-ganization.

Goalsettingalsotendstofollowahierarchicalpatternthroughoutanorganization. However,incontrasttogoalsetting,theremaynotbeaone-to-onerelationshipbetweenCSFsastheycascadethroughthevariouslayersoftheorganization. ThisisbecauseCSFsareoftencloselytiedtoaparticularmanagerormanagementlayerandanyspecificconcernsatthatlevel. Thus,theremaybesomeCSFsatlowerlevelsintheorganizationthatareimportanttoachievinghigherlevelCSFsandaccomplishingtheorganizationsmissionbutarenotexplic-itlyrelatedorsubordinatetoahigherlevelCSF.

Figure10:ExampleofHierarchyofCSFsinanOrganizationInourexperiencewithCSFs,wehavefounditusefultodescribetwolevelsofCSFs:enter-

priseCSFsandoperationalunitCSFs.

4.5.1Enterprise14CSFsThe

numerous

sources

of

CSFs

illustrate

the

broad

array

of

challenges

and

demands

facing

managementinmodernorganizations. Eachlayerofmanagementhasasetofconditionsthatmustbemonitoredandactedupon. TheyalsohaveauniquesetofCSFstoconsider.

14 RockhartreferstothesetypesofCSFsgenericallyas corporateCSFs becauseofthefocusofhisworkonthecorporateworld. However,throughoutthisreport,andparticularlyinthecasestud-ies,weusetheterm enterpriseCSFs wheneverwemakeageneralreferencetothecriticalsuc-cessfactorsforanorganization.


41/135

CMU/SEI-2004-TR-010 25

ButasimplegatheringoftheCSFsofeachmanager(andmanagementlayer)intheorganiza-tiondoesnotnecessarilyformasupersetofenterpriseCSFs. ThisapproachcouldresultinhundredsorpossiblythousandsofCSFsthatthehighestlevelsofmanagementwouldneedtoconsider. (Imaginethedifficultiesthatstrategicplanners,forexample,wouldhaveinat-temptingtoaligntheirplanningactivitieswithhundredsofCSFs.) Itcouldalsoderailtheorganizations

ability

to

focus

on

those

five

to

seven

areas

that

can

truly

make

or

break

theireffortstoaccomplishthemission.Aswithothermanagersintheorganization,executive-levelmanagersmustbeguidedbytheirownsetofuniqueCSFs. However,becauseoftheroleofexecutive-levelmanagement,theirCSFsalsotypicallyrepresenttheorganizationstrulycriticalandkeyareasofperform-ance. ThisisnottosaythattheCSFsofotherlayersofmanagementarenotimportantexecutive-levelmanagersstrategicdirectionstronglyinfluencestheCSFsofotherlayersofmanagement,andtheirabilitytoachieveenterpriseCSFsishighlylinkedtosuccessinachievinglowerlevelCSFs.Thus,anorganizationcandevelopahigh-levelsetofCSFsthatrepresentthetopactivities,concerns,strategies,andgoalsofexecutive-levelmanagement.These enterpriseCSFs arederivedfromthetoptwoorthreelayersofmanagementandreflectthevariousCSFsfoundthroughouttheorganization. InourworkwithCSFs,wehavefoundthatenterpriseCSFs

providethemosteffectivestrategicviewofwhatisimportanttotheorganizationandtoac-complishingtheorganizationsmission. EnterpriseCSFsrepresenttheentireorganization,andeachoperationalunitinsomewaycontributesto(ordetractsfrom)achievingthembyachievingitsoperationalunitCSFs.4.5.1.1 NatureofEnterpriseCSFsEnterpriseCSFsoftenreflectboththecurrentconcernsofexecutive-levelmanagersaswellasthelongertermstrategicdirectionoftheorganization. Asaresult,enterpriseCSFscancompriseablendoftemporalCSFs(reflectingthecurrenthotissues ofmanagement)andindustry,peer,andenvironmentalCSFs(whichreflectsuchindicatorsasthestateoftheeconomy,currentbusinessclimate,andgeopoliticalissues). Thisisimportantbecauseexecu-tive-levelmanagersoftenmustbeagileandabletoreacttochangesinadditiontoplanningforthelongrun.

4.5.2OperationalUnitCSFsAnoperationalunitcanbedescribedasanorganizationaldepartment,division,subdivision,oranyothergroupingofactivitiesthatshareacommonfunction,purpose,ormission. Forexample,thefinancedepartmentinanorganizationmightbeanoperationalunit.Regardlessofhoworganizationsdefinetheiroperationalunits,eachmayhaveitsownsetofCSFs.


42/135

26 CMU/SEI-2004-TR-010

AsnotedwithenterpriseCSFs,operationalunitCSFsarenotnecessarilyasimplecollectionoftheCSFsofmanagersintheoperationalunit. Instead,operationalunitCSFsmayreflecttheconcernsandstrategicdirectionofseniormanagersintheunit,aswellasthestrategicdirectionoftheorganization(asembodiedinenterpriseCSFs).

ItisimportantnottoconfuseoperationalunitCSFswithmanagement-functionCSFs. Man-agement-functionCSFsreflectthegenericresponsibilitiesthatareinherentinthemanagers

positionintheorganization. Incontrast,operationalunitCSFsaresimilartoenterpriseCSFsinthattheyreflecttheoperatingperspectiveandstrategicdirectionofexecutive-levelmanag-ersintheoperationalunit. ThemanagementlayeriscertainlyasourceofCSFsfortheopera-tionalunitbutisnotentirelyreflectiveofit.4.5.2.1 NatureofOperationalUnitCSFsInourdefinition,operationalunitCSFstendtobelessinfluencedbytheorganizationsindus-tryandmorefocusedonthecontributionsnecessarytosupporttheorganizationsstrategicgoalsandmission. Forexample,intheairlineexample,theoperationalunitCSFsforfourdivisionsordepartments reservations,scheduling,flightoperations,andfreightopera-tions areverydifferent,buteachcontributesvitallytotheorganizationsoverallachieve-ment.OperationalunitCSFsmayalsohaveatemporalcomponent,particularlyifaspecificdivi-sionintheorganizationhastemporarychangesinoperatingconditionsthatitmustconsider.Forexample,iftheairlineindustryasawholemustcontendwithovercapacity,the schedul-ing departmentmayhaveaCSFthatseekstoreduceflightsanddestinationsserveduntildemandincreases.

4.5.3RelationshipBetweenHierarchyandSourceEachofthesourcesofCSFs(industry,environment,etc.)cansupplyCSFsattheenterpriseoroperationalunitlevel. However,becauseoftheirnature,somesourcesaremorelikelytosupplyCSFsateithertheenterpriseoroperationalunitlevels. Forexample,industryCSFsmaysupplymoreCSFstotheenterpriselevelthantotheoperationalunitlevel. Table1summarizesthepossiblerelationshipsbetweenenterpriseoroperationalunitCSFsandthevariousCSFsources.


43/135

CMU/SEI-2004-TR-010 27

Table1:

MatrixofCS

FLevelstoCSFTypes

TypeofCSF

CSFLevel

Industry

Peer

Environmental

Temporal

Management-

Function

Enterprise

IndustryCSFsstrongly

influenceenterpriseCSFs.

Executive-levelmanagers

haveadirectresponsibility

forinteractingwiththe

externaloperatingenvi-

ronmentoftheorganization

asreflectedinindustry

CSFs.

Executive-levelmanagers

mustbemindfulofthe

competitive

positionofthe

organizationandcalculate

theirroleto

ensurethey

planaccordingly.

Factorssuchasseasona

lity

andthecurrentgeopolitical

environmentaffectthe

currentandlong-termp

lans

oftheorganization.Ex

ecu-

tive-levelmanagersmu

st

considertheimpactofthe

environmentontheirstra-

tegicplans.

Atemporaryproblemor

changeintheorganiza-

tionsstrategycanaffect

theoverallCSFsforthe

organization.Thehottest

issuesforexecutive-level

management(suchassecu-

rity)mustbeconsidered

andaddressed.

EnterpriseCSFsreflectthe

uniq

ueresponsibilitiesof

executive-levelmanagers.

Theirpositiongenerally

reflectstheiruniqueroles,

suchasriskmanagement,

financialmanagement,and

shareholderinteraction.

OperationalUnit

IndustryCSFscouldinflu-

enceoperationalunitCSFs,

especiallyifaparticular

divisionisaffected.How-

ever,onthewhole,thereis

lessfocusontheindustryat

thislevelthanattheorgan-

izationallevel,particularly

iftheoperationalunitis

fairlylowintheorganiza-

tion.

Operationalunitsmayhave

lessrespons

ibilityforthe

competitive

positioningof

theorganization;therefore

thismaynotbeasourceof

CSFs.However,ifthe

operational

unitisadivi-

sionthatcompetesina

uniqueindu

stry,competi-

tivepositionCSFswill

arisesimilartothosethat

couldbefoundattheor-

ganizationallevel.

Environmentalfactorsmay

filterdowntoanopera-

tionalunit,particularly

ifit

isadivisioncompeting

ina

uniqueindustry,resulting

insomeenvironmental

CSFs.

Temporaryproblemsor

changesaffectingtheor-

ganizationasawholemay

filterdowntoanyopera-

tionalunitthatiscriticalto

dealingwiththeseprob-

lemsorhelpingtoimple-

mentchanges.Therefore,

sometemporalCSFsmay

befoundattheoperational

unitlevel.

Ope

rationalunitCSFsare

high

lyinfluencedbyman-

agementlayerCSFs.Op-

erationalunitstendtore-

flectmanydifferentunique

laye

rsofmanagement

(middle,line,etc.)and

ther

eforearearichsource

ofm

anagement-function

CSF

s.


44/135

28 CMU/SEI-2004-TR-010

4.5.4OtherConsiderationsEnterpriseandoperationalunitCSFsmustfittogetherandrelatetooneanother,buttheyaregenerallymuchmorelooselycoupledthangoals. Goalstendtocascadethroughouttheor-ganizationsothatthereisatightone-to-onefitbetweenthegoalsofeachmanagementlayer.Forexample,thegoalsofaproductionlineworkeraredirectlyrelatedtothegoalsofthepro-ductionlinemanager,whosegoalsinturnarefocusedonhelpingtoachievethegoalsofthechiefoperatingofficerandtheorganization.ThestrictbalancingandlevelinginherentingoalsettingisnottypicallyfoundwithCSFs.Theremaynotbeaone-to-onematchbetweeneveryoperationalunitCSFandanenterpriseCSF. Thisisbecauseeachlayeroftheorganizationhasitsownfocusandoperatingcondi-tions,includingexecutive-levelmanagement. However,theremustbecongruence;otherwisetheremaybeadisconnectionbetweenwhatanoperationalunitviewsasimportantandwhatisgoodforthelargerorganization.

Figure11:RelationshipBetweenEnterpriseandOperationalUnitCSFs


45/135

CMU/SEI-2004-TR-010 29

5 ApplyingCSFs

Atthecore,CSFsrelatetothefunctionsofmanagement15 whatneedstobedone,howwell,andhowoftentomeetapersonalororganizationalmission. Intheirsimplestform,CSFscan

beviewedasamanagementtoolformakingbetter-educateddecisionsthatconsciouslysup-portthemissionoftheorganization. Infact,applyingCSFstovalidateandensurealignmentwiththedirectionandintentoftheorganizationcanenhanceanydecision,initiative,effort,orprocess.Inthissection,wedescribethetraditionalusesofCSFsandsomegeneraladvantagesofaCSF-basedapproachtoorganization-wideeffortsandinitiatives. Mostimportantly,weex-

plorethepotentialbenefitsoftheCSFmethodasspecificallyrelatedtoaddressingsecuritystrategy,goals,andactivities. Finally,otherpotentialusesofthemethodthatwebelievemeritfurtherresearchandfieldtestingarepresented.

5.1 HistoricalApplicationofCSFsAsnotedinSection3.1,muchofthecontemporaryliteratureregardingCSFs(certainlythatwhichpostdatesRockhartsintroductionoftheCSFapproachintheHarvardBusinessRe-view[Rockhart79])focusesontheconnectionbetweenCSFsandinformationsystemsandtechnology. Eventhecreatoroftheconcept,D.RonaldDaniel,hadinformationsystemsinmindwhenhecoinedthephrase successfactors andcreatedtheconceptthatRockharteventuallytransformedintoCSFs. Ironically,Danielsunderlyingobjectivewastohelpor-ganizationsmanagemoreeffectively;however,hequicklyacknowledgedthatthiswasin-creasinglydependentonhigh-qualityinformationandtechnology. Thus,thebondbetweenCSFsandinformationsystemswascreatedandhascontinuedtoevolve.

15 HenriFayolsclassicviewofmanagementincludesthefunctionsofplanning,organizing,com-manding,coordinating,andcontrolling. Theeffectivenessofeachofthesefunctionscanbegreatlyenhancedifperformedwithinthecontextoftheorganizationscriticalsuccessfactors.MoreinformationonFayolsmanagementfunctionscanbefoundathttp://www.onepine.info/.


46/135

30 CMU/SEI-2004-TR-010

5.2 GeneralAdvantagesofaCSF-BasedApproachThroughoutthisreport,theadvantagesofdevelopingandapplyingCSFsarepresented. Theseeminglyendlesswaysinwhichtheycanbeofusetoanorganizationspeakstotheirsimplenatureandbroadapplicability.OfnoteisRockhartsviewthatoneofthemostpowerfulusesofCSFsistoenhancecommu-nicationamongtheorganizationsmanagers[Rockhart79]. Theabilitytogetmanagers onthesamepage canaidinmobilizingallareasoftheorganizationtowardthesamegoals.RegardlessofhowCSFsareused,thereareseveraladvantagestohavingthistypeofcom-monfocusfortheorganization: CSFscanreduceorganizationalambiguity. Developingandcommunicatingasetof

CSFscanreducethedependenceontheperceivedaimsoftheorganization. CSFsreflecttheimplicit,collectivedriversofkeymanagersandasaresultareamoredependableandindependentarticulationoftheorganizationskeyperformanceareas.

CSFsaremoredependablethangoalsasaguidingforcefortheorganization. Anorgani-zationcansetgoodgoalsthat,intheory,willmovetheorganizationtowarditsmission.However,ifthegoalsarepoorlyarticulatedordeveloped,thisisnotguaranteed. CSFsarereflectiveofwhatgoodmanagersdowelltomovetheorganizationtowarditsmis-sion,regardlessofthequalityofthegoalsthathavebeenset.

CSFsaremorelikelytoreflectthecurrentoperatingenvironmentoftheorganization.Goalsettingtendstobeacyclical(i.e.,yearly)activitythatisseldomrevisiteduntilper-formancemeasurement. Usedproperly,CSFsarelikelytobemoredynamicandtore-flectcurrentoperatingconditions(particularlybecauseofthemanysourcesofCSFs).

CSFsprovideakeyrisk-managementperspectivefortheorganizationtoconsider. Theriskperspectiveofexecutive-levelmanagersisbuiltintoCSFs,sotheir radarscreen isexposedtotheorganizationasawhole.

CSFscanbevaluableforcoursecorrection. WhenCSFsaremadeexplicit,managersoftenrealizethattheirperceptionofwhatisimportanttotheorganizationmaynotmatchrealityortheymayrealizethattheydontfullyunderstandthecurrentoperationalcli-mate. Thus,theycanuseCSFstorealigntheiroperatingactivities.

5.3 UsingCSFsinaSecurityContextOurinterestintheCSFapproachevolvedfromourrecurringobservationthatcustomersof-tenhavedifficultydevelopingandimplementingasecuritystrategywhentheydonotmain-tainanexplicitfocusonbusinessdrivers. Thiscanoccurforanumberofreasons: Theorganizationmayhavedecidedthatsecurityisthedomainoftheinformationtech-

nologydepartment,whichmaynotplayastrategicroleorisunabletoarticulatetheover-allgoalsoftheorganization.


47/135

CMU/SEI-2004-TR-010 31

Securityisviewedasacostorburdenthatmustbemanagedandnotasanactivitythatcontributestosuccess,profitability,orgrowth.

Personnelinchargeofsecurityaredisconnectedfromtheorganizationsmissionbecauseoftheirroleorfunction(i.e.,theyareexternaltotheorganization,aswithconsultants,ortheyhaveastricttechnologyfocus)orbecauseofthelayeroftheorganizationwheretheyoperate(i.e.,stafforlinefunctions).

Theorganizationsbusinessdriversorfactorsforsuccesssimplyarenotwellknownorcommunicatedtoallwhohaveaneedtoknow.

Regardlessofthereason,theresultisoftenthesame:thesecuritystrategyfailstoreflectwhatsimportanttotheorganization,totheaccomplishmentofitsmission,andtoitslong-termresiliency. Itfailstoanswerthebasicquestions: Whatistobeprotected? Howisitthreatenedorwhydoesitneedtobeprotected? Whathappensifitisnotprotected? Cer-tainly,thesequestionsarefundamentaltoariskmanagementapproachtosecurity,buttheanswersareoftenembeddedintheorganizationsmission,goalsandobjectives,andthefac-torsthataffecttheorganizationspotentialsuccessorfailureinpursuitofthemissionandgoals theCSFs.Unfortunately,manyorganizationswithwhomwehaveworkedhaveonlyavagueunder-standingoftheirCSFs. Theyoftenrelyontheirperceptionof important or critical ratherthanrelyingonanexplicitarticulationofthesefactors. Theyalsotendtorelyonexternalinfluences(suchaslawsandregulations)toprovidethemwithadefaultsecuritystrategyorinitiativeinsteadofdevelopinganinternalstrategy,consistentwiththeirmission,thatcan

positionthemtoaddressever-increasingandchangingregulations.Overall,itisourcontentionthatorganizationsthathaveacleareyeontheprize arebetter

positionedtomakemeaningfuldecisionsaboutsecurityandtoimplementtheminawaythatnotonlyprotectstheorganizationbutactuallycontributestotheaccomplishmentofthemis-sion. Properlypositionedandmanaged,organizationscanturntheburdenofsecurityintoacompetitiveadvantage anenablerthatdirectlyaffectsanorganizationsachievementofitsgoalsanditsbottomline. Someorganizationshavehadtoadoptthisperspectiveonsecurity

becauseitisrequiredbythenatureoftheindustryinwhichtheycompete. Forexample,thebusinessmodelformanye-commerceorganizationsisbuiltontrustandsecurity. Thus,theirsecuritystrategyisinextricablylinkedtotheirmission ifthestrategyiseffective,theymeettheirgoals;ifnot,thebottomlinesuffers.Inthissection,weprovidesomeofourtheoriesandshareourexperiencesregardingtheuseoftheCSFmethodtoenabletheeffectivedevelopmentofsecuritystrategyandtheapplica-tionandmanagementofsecuritythroughoutanenterprise.


48/135

32 CMU/SEI-2004-TR-010

5.3.1EnterpriseSecurityManagementSeveralyearsago,wewerecalledupontoassistafederalgovernmentagencyinitssecurityefforts. Theagencyhadrecentlydecidedtodevelopitsowninformationsecuritycapability,throughwhichitwouldnotonlyserveitselfbutseveralotherhigh-profilegovernmentagen-cies. Ourscopeofworkwastoperformariskassessmentfortheagencytoidentifytheis-suesthatitwouldneedtoaddressfirst. However,itsoonbecameclearthatariskassessmentactivitywouldnotanswersomeofthebasicquestionsandissuestheagencyneededtocon-front.Ateamwithabroadarrayoftechnologyandsecurityskillswasassembledtostafftheinfor-mationsecuritycapability.However,whattheagencyhadintermsofhumanresourcesdidnotcompensateforwhatitlackedinotherkeyingredientsforsuccess therewasnoexistingsecuritypolicyorstrategy,nosharedvisionorobjectivesforstrategyacrossthevariousagencies,and,moreimportantly,noclearvisionofwhatitwantedtoaccomplishandwhy. Inaddition,theteamappearedtolackclarityonitsroleandresponsibilities.Ourworkpromptlytooktheformofhelpingtheteamtodetermineitssecuritygoalsandob-

jectivesandtotakeaninventoryofitsstrengthsandchallenges. Theteammembersunder-stoodthattheyneededto securetheorganization butwerenotabletoclearlyarticulatethemeaningof secure and,further,howtheywouldknowwhentheyhadaccomplishedit.Weobservedthat,asanewlyformedgroup,oneoftheirmajorchallengesindefining se-cure or security wasthattheteamlackedcontext membershadnocomfortorfamiliaritywiththemissionofthelargeragencyorthemissionsoftheotherverydiverseagenciesthattheywerechargedtoprotect. Beforeourworkprogressedanyfurther,wesuggestedthatitmightbeagoodideatocollecttheseagenciesmissionstatementsandstudythemtogetasenseofwhatwasimportant. Thisinformationcouldthenhelptodeterminethecapabilitiesthattheteamwouldneedtomeetitsrequirementsformanagingsecurityacrosssuchavastenterprise.Inhindsight,whatwewereattemptingtodowastogettheagencytosetthecontextforitssecurityefforts todevelopaguiding position ora posture aswedescribeditatthetime.Wepromptedtheagencytolookclearlyandexplicitlyatthedriversusedbytheorganizationtoaccomplishitsoperationalgoalsandtoalignitssecuritystrategiesandactivitiestothosedrivers. Inthatway,agencypersonnelmightnotonlybesupportingbutcontributingtotheoperationalgoalsthroughtheirwork. WhilewedidntperformaCSFexercisewiththeagency,itbecamecleartousthatinthefuture,thistypeofexercisewouldbeavaluablecon-text-settingexerciseforcustomersfacingsimilarproblems.Italsobecameapparentduringourengagementthatthesmallsecuritystaffthattheagencyhadassembledwouldnotbeabletoaccomplishitssecuritygoalsalone. Itwouldneedto


49/135

CMU/SEI-2004-TR-010 33

drawuponandmobilizeexistingcapabilitiesoftheorganization,bothtechnicalandmanage-rial,tobesuccessful.5.3.1.1 EnterpriseSecurityManagementDefinedOurexperiencewiththisfederalgovernmentagency(andsubsequentlyseveralotherorgani-zations)evolvedintoamanagement-andprocess-orientedviewofsecurityasabusiness

processthatispervasiveacrossanddependentontheenterprise.OurcontinuingexplorationofthesetheoriesisthefocusofanemergingbodyofworkintheNetworkedSystemsSurviv-abilityprogramattheSEI,referredtoasenterprisesecuritymanagement(ESM). Thecoreassertionofthisworkisthatmanagingsecurityacrossanenterpriseisacomplexendeavorthatdependsonseveralfundamentalprinciples: Theskills,capabilities,andeffortsoftheentireorganizationmustbeutilizedandmobi-

lized. Keyfunctionsandprocessesintheorganizationmustcollaborateonsharedsecuritygoals

andstrategy. Theorganizationssecurityobjectivesoranarticulationofits desiredstate mustbede-

velopedandunderstood. Criticalassetsthatareessentialtoachievingtheorganizationsmissionmustbeidentified

andprotected. Informationtechnologyoperationsandsupportmustenablesecuritygoals.Oneofthekeystoachievingsuchanextensiveundertaking,particularlywheremanydiverse

partsoftheorganizationmustworktogether,istoensurethatitisproperlyfocusedonasharedunderstandingoforganizationalvalues suchasCSFs.5.3.1.2 ESMandCSFsThecomplexityofundertakinganenterprise-wideviewofsecuritymanagementcanbeillus-tratedinthechallengesfacingchiefsecurityofficers(CSOs). Often,CSOsaretaskedwith securing theorganization,butmaynotbeclearonwhatthatmeans. Indeed,insomeor-ganizations,theroleoftheCSOhasbeenrelegatedtotheinformationtechnologydepart-ment,furtherseparatingitfromorganizationalstrategyandbusinessdrivers. Asaresult,theCSOisoftenlefttoanswersomeveryimportantorganizationquestionswithoutspecificguidance: Whatneedstobesecured?Why,andinwhatpriority? Whatpartsoftheorganizationmustbeinvolvedinthiseffort? HowwillIconvincethese

unitstoworktogether,especiallyifIdonthavedirectcontroloverthem? HowwillIknowwhentheorganizationhasbeen secured? Whatwillbeusedtomeas-

uresuccess?


50/135

34 CMU/SEI-2004-TR-010

Ourassertionisthatsomeoftheanswerstotheseimportantquestionsarefoundintheor-ganizationsbusinessdrivers,andinparticularitsCSFs,becausetheyrepresentacommon,sharedfocus. Why? The fieldofvision oftopmanagement(andmanagementingeneral)isrepresentedin

CSFs. Thisprovidesapowerfulclarificationofwhatisimportantandvaluedintheor-ganization. FailuretoachieveCSFsdirectlyaffectstheorganizationsabilitytoaccom-

plishitsmission. Thus,securityeffortsneedtoalignwithCSFsandensurethattheac-complishmentofCSFsisnotimpeded.

CSFsreflectthegoalsoftheorganization. Managersoperatetowardtheachievementofgoals. Whatneedstobeprotectedintheorganizationcanbeidentifiedrelativetothesegoals assetsandprocessesthatsupportthesegoalsandtheorganizationsmissionmust

beprotected. Rallyingaroundacommonpurposeisaneffectivemeansforgettingdisparatepartsof

theorganizationtotakeonacommoncause,suchassecurity. Securityisabusinessproblemthatrequirestheeffortofeveryoneintheorganizationtosolveandtomanage.CSFsprovideaunifyingeffect,ifonlybecausemostemployeesprefertoavoidthestigmaoffailingtocontributetoaneffortthatisclearlygoodfortheorganization.

Thedriversforsecurityshouldbethesameasthebusinessdriversusedbytheorganiza-tiontoaccomplishitsmission. Securityshouldbeawayfororganizationstoenhancetheiroperations,helpthemachievetheirgoals,andprovidethemwithanappropriatelevelofresiliencycommensuratewiththeirlong-termstrategies. CSFscanbeshareddriversforsecurityandtheorganization.

Forthesereasons,weseegreatpromisefortheCSFmethodasacatalystforsettingthedirec-tionofanorganizationsenterprisesecuritymanagementactivities. ChiefsecurityofficerscanconfrontthechallengesofenterprisesecuritymanagementbyusingCSFsasafoundationfromwhichsecurityprofessionalsandtherestoftheorganizationcancollaborate,plan,andexecute. Theycanalsoqualitativelymeasurethesuccessoftheirsecurityprogramsbyde-termininghowtheycontributetoachievingtheorganizationsenterpriseCSFs.

5.3.2InformationSecurityRiskAssessmentandManagementOneofthekeyactivitiesinmanagingsecurityistoperformperiodicriskassessments. Ingeneral,riskassessmentsareadiagnostictoolthathelpstheorganizationtodeterminethesuccessofitssecurityeffortsrelativetoitssecuritystrategy. TheCSFmethodshowsparticu-larpromiseinhelpingorganizationsconductmoremeaningful(andvalid)informationsecu-rityriskassessmentsinanumberofareas.


51/135

CMU/SEI-2004-TR-010 35

MostofourfieldworkexperienceininformationsecurityriskassessmentisintheuseandapplicationoftheOCTAVE16method. TheOCTAVEmethodprovidesspecificguidanceforthemajoractivitiesofariskassessment,butalsoallowsforsignificanttailoringtomeettheneedsofuniqueorganizations. Asaresult,manyuserswithwhomwehaveworkedhaveaskedusforadditionalguidanceondevelopingscope,selectingcriticalassetstoassess,andin

prioritizing

risks

to

mitigate.

Without

the

advantage

of

the

CSF

method,

we

often

pro-

vidednospecificguidancetocustomersexcepttoencouragethemtoalignriskassessmentactivitieswithbusinessdrivers. However,theterm businessdrivers isoftenambiguousandsubjecttointerpretation. Unlessanorganizationhasacleardefinitionofitsbusinessdrivers,theycannotbeusedinapracticalwaytoguideimportantorganizationaleffortsorinitiatives.Becauseofthisissue,webegantosearchforamorepreciseandpracticalwaytoapplytheconceptofbusinessdriverstosecurity. Throughfurtherresearchandfieldwork,wedecidedtoexploretheuseofCSFs. CSFsareinextricablylinkedtoandrepresentativeoftheothercomponentsofbusinessdrivers(i.e.,theorganizationsmission,values,andpurposeanditsgoalsandobjectives).CSFsarealsoaconduittoachievingtheorganizationsgoalsandob-

jectivesandaccomplishingitsmission. Thus,theuseofCSFscanbeaneffectivewaytolinkbusinessdriverstovariousaspectsofsecurity,includingdevelopingandimplementingsecu-ritystrategy,managingsecurityactivitiesandoperations,andconductingsecurityriskas-sessments. Onthispremise,thefollowingsectionshighlightthewaysinwhichCSFscanenhancekeyriskassessmentactivities.5.3.2.1 DeterminingRiskAssessmentScopeOneofthemostimportant(anddifficult)tasksinperformingariskassessmentistodeter-mineitsscope.Ariskassessmentperformedonanareaoftheorganizationthatisnotessen-tialtoaccomplishingthemissiongenerallywillnotyieldmeaningfulresults. Unfortunately,failingtoproperlyscopetheriskassessmentalsodiminishesthepurposeandintentofusingarisk-basedapproach.Forexample,theOCTAVEmethodforriskassessmentguidesuserstochoosethreetofiveimportantoperationalareastoincludeinthescope. Thisguidanceisperfectlyacceptableforuserswhohaveagoodsenseoftheorganizationsmissionandcanbeobjectiveaboutwhichareascontributemosttoaccomplishingthemission. However,formanyusers,particularlythoseinthelowerlevelsoftheorganization,thisguidanceisdifficulttoputintopractice.Frequently,usersneedanexplicitsetofcriteriaagainstwhichtoevaluateoperationalareasandtodecidewhichareasshouldbeincludedintheriskassessment. CSFsareusefulforthis

purposebecausetheyrepresenttheorganizationsbusinessdriversandtheyembodytherisk-managementperspectiveofexecutive-levelmanagement.

16 MoreinformationontheOCTAVEmethodcanbeobtainedfromhttp://www.cert.org/octave.


52/135

36 CMU/SEI-2004-TR-010

UsingCSFs,anaffinityanalysis17canbeperformedbetweenenterprise(oroperationalunit)CSFsandthevariousdepartmentsoroperationalareasoftheorganizationbeingconsideredforassessment. ThoseoperationalareasthatprovidesignificantsupportfortheachievementofCSFswillbestrongcandidatesforriskassessmentbecauseoftheimpliedcontributiontheymaketowardaccomplishingtheorganizationsmission.Figure12providesanexampleofthepossibleintersectionsbetweenenterprisedepartmentsandCSFsforthepurposeofidentifyingareasinwhichtoperformariskassessment.

Figure12:AffinityAnalysisforDeterminingISRMScope

17 ThetechniqueusedtoperformaffinityanalysisisprovidedinAppendixA,CSFMethodDescrip-tion.


53/135

CMU/SEI-2004-TR-010 37

5.3.2.2 SelectingCriticalAssetsforAssessmentArisk-basedapproachtosecurityencouragesorganizationstodirecttheirlimitedresourcesto

protectingtheorganizationsmostcriticalassets informationandtechnical18assetsthatareessentialtosupportingtheorganizationsmission. Theselectionofcriticalassetsforriskas-sessmentisoftenlefttothejudgmentofthoseperformingorparticipatingintheassessment,whethertheyareinsideoroutsideoftheorganization. Thustheimportanceoftheassetmay

bebasedonitsperceivedvalue,ratherthanamoreconcretemethodofassetvaluation.Whiledesirable,assigningaqualitativeorquantitativevaluetoassetsmaybeprohibitivelyexpensiveforanorganization.TheuseofCSFscanbeasimpleyeteffectivecompromiseforselectingcriticalassets. Asa

byproductofusingCSFstohelpdefinethescopeofariskassessment,thepoolofpotentialassetscanbeeffectivelylimitedtothoseoperationalareasthataremostimportant. Con-versely,fororganizationsthathaveasolidinventoryofinformationandtechnicalassets,af-finityanalysiscanbeperformedtocompareassetstoCSFs. Theresultofthistypeofanaly-sisistheidentificationofassetsthatareessentialtoachievingCSFsand,bydefault,toaccomplishingthemissionoftheorganization. Insummary,CSFscanhelptovalidatetheimportanceofanassetbyconfirmingitsoverallsignificancetotheorganization.Figure13portraysanexampleofaffinityanalysisbetweencriticalassetsandasetofenter-

priseCSFs. Inthiscase,thereisanintersectionbetweenthe financialdata assetandthe managecompliance CSF.Thisindicatesthatthe financialdata assetiscriticaltotheor-ganizationbecauseitisessentialtoachievingthe managementcompliance CSF,andthusneedstobeprotected.

18 Informationassetsrepresentthedataandinformation,ineitherphysicalorelectronicform,thatiscriticaltotheorganization. Technicalassetsrepresentthoseassetsthatsupportthestorage,trans-mission,andprocessingofdataandinformationandthereforeareimportanttotransformingdataandinformationforusebytheorganization. Peoplecanbeanassettotheorganizationaswellforsimilarreasons theycanbeaprimarywayofstoring,transporting,orprocessingdata.


54/135

38 CMU/SEI-2004-TR-010

Figure13:AffinityAnalysisforDeterminingCriticalAssets5.3.2.3 IdentifyingandValidatingSecurityRequirementsAnimportantcomponentofprotectingcriticalassetsisthedevelopmentofsecurityrequire-mentsintheareasofconfidentiality,integrity,andavailability.19 Asanassetisstored,trans-

ported,andprocessedthroughouttheorganization,thesesecurityrequirementsmustbemetandprotectedbyallwhouseortakecustodialcontrolofassets. Definingsecurityrequire-mentscanbeadifficulttask;significantthoughtmustbegiventothepotentialmisuseoftheassetsandtheconsequencesofthismisuse. Inaddition,asubstantialnumberofrequirementscouldbedevelopedforeachasset. Thisposesaproblemfordevisingaprotectionstrategyforanasset: Whichrequirementsaremostimportant? Whichrequirements,ifunmetforanyreason,wouldimpacttheowneroftheassetortheorganizationasawhole? Further,whichassets,ifimpaired,wouldimpacttheachievementofCSFs?Answeringthesequestionsrequiresconsiderationofthepriorityofthesecurityrequirements.CSFscanbeveryusefulforthispurposebecausetheyrepresentmanagementspriorities.Forexample,acomparisonofanassetssecurityrequirementstoCSFswillhighlightthoserequirementsthatareessentialtoensuringthattheachievementofCSFsisnotimpeded. Pri-oritizingrequirementsinthismannercanhelptheorganizationtodevelopandimplement19 Securityrequirementsinthesecategoriesarecommonlyappliedonlytoinformationassets. Tech-

nicalassetshavesecurityrequirementsaswell,butarenotoftendescribedintermsofconfidenti-ality,integrity,oravailability.


55/135

CMU/SEI-2004-TR-010 39

meaningfulsecuritycontrolsforassetstoensurethattheycontinuetocontributetotheor-ganizationspursuitofitsmission.Figure14providesanexampleofaffinityanalysisforsecurityrequirements. Inthisexam-

ple,thesecurityrequirementof confidentiality forthe medicalrecords assethasbeenidentifiedasimportanttothe managecompliance CSF. Thisisbecausefailuretomeettheconfidentialityrequirementformedicalrecordscouldimpedetheorganizationsabilitytobesuccessfulatmanagingcomplianceactivities.

Figure14:AffinityAnalysisforDetermining/ValidatingSecurityRequirements5.3.2.4 IdentifyingRiskstoCriticalAssetsRiskidentificationisatthecoreofarisk-managementapproachtosecuringcriticalassets.Properlycharacterizingariskisessentialtounderstandingthepotentialimpactontheownersoftheassetifitissomehowcompromised,temporarilylost,orpermanentlydestroyed.Whilethistaskisessential,itcanalsobethemostelusiveforanorganizationtoundertake.Asnotedpreviously,definingthescopeofariskassessmentanddeterminingthecriticalas-setsonwhichtofocustheassessmentisanimportantfirststep. However,theorganizationstillhastodecideuponwhichriskstodirectlimitedresources. Todothis,anorganizationhastwooptions:1. Useageneralizedtaxonomytoidentifyrisk. Thisapproachispopularwithfederalgov-

ernmentagenciesandisofteneffectivebecauseitprovidesanorderlyandsomewhatcomprehensiveguideforexaminingmanypotentialareasofrisk.


56/135

40 CMU/SEI-2004-TR-010

2. Elicitriskinformationdirectlyfromtheorganization. ThisistheapproachusedbytheOCTAVEmethodand,dependingontheorganization,canalsobeveryeffective. Itat-temptstoensurethattheexperienceandintuitionofmanagersandstaffintheorganiza-tionisreliedontoidentifyrisksthataremostassociatedwiththebusinessdriversoftheorganization.

Whileeffective,therearepotentialproblemswitheachoftheseapproaches. Forexample,exclusivelyusingataxonomymaycausetheorganizationtooverlookcertainrisksthatareuniquetoitsbusinessenvironmentortospendvaluabletimeconsideringriskstowhichitisnotspecificallyexposed. Inaddition,successinusingaknowledgeelicitationapproachishighlydependentonensuringthattherightparticipantsareinterviewedandthattheyfullyunderstandtheriskassessmentapproachandobjectives. Whileitmaybeeffectiveinidenti-fyingrisksthatareuniquetotheorganization,thisapproachcanresultinoverlookingmanycommonrisksthattheparticipantsarenotfamiliarwithbecausetheyhavealimitedunder-standingofinformation,technical,andphysicalsecurityissues. Thus,theresultsfromthisapproachareonlyasgoodasthequalityoftheparticipantsintheprocess.OnewaytoenhancetheeffectivenessofeitheroftheseapproachesistouseCSFs. Forex-ample CSFscanbeusedtoproperlyfocusriskidentification. Withataxonomyapproach,CSFs

canhelptofocusinonthoseareasofthetaxonomythatdirectlyaffect(encourageorim-pede)theaccomplishmentofCSFs. Inthisway,thetaxonomyismoreeffectivelylinkedtotheorganizationsbusinessdriversandareasthatareunimportanttotheorganizationarenotconsidered.

Inthecaseoftheknowledgeelicitationapproach,CSFscanbeaverypowerfulmeansforshapingandguidingtheresponsesofparticipants. Knowledgeofenterprise(oropera-tionalunit)CSFscanenableparticipantstoidentifyareasofconcernandrisksthatex-

plicitlyconsiderthepotentialimpactonachievingCSFs. Inthisway,theparticipantsareprovidinginformationthatismorecertainlylinkedtotheorganizationsbusinessdrivers.(ThisisillustratedinthecasestudypresentedinAppendixB.)

Likewise,onceriskshavebeenidentified,CSFscanbeusedforvalidation. RiskstocriticalassetsthatdonotimpairtheachievementoftheorganizationsCSFsmaybegivenalowerprioritybecausetheyareunlikelytoimpacttheorganizationsabilitytoac-complishitsgoalsandmission. Asaresult,risksthatinterferewiththeorganizationsability

to

achieve

CSFs

can

then

be

focused

on

because

they

have

the

greatest

potential

forharm.

5.3.2.5 SettingEvaluationCriteriaforMeasuringRis

Critical Success Factors 0407

Documents

Transcript of Critical Success Factors 0407