Open Testbeds deliverable-final - RITICSritics.org/wp-content/uploads/2018/07/Open... · PETRAS The...
16
Open Testbeds for CNI Authors Chris Hankin (editor), Imperial College London Deeph Chana, Imperial College London Ben Green, Lancaster University Rafiullah Khan, Queen’s University Belfast Peter M3, National Cyber Security Centre Peter Popov, City University of London Awais Rashid, Lancaster University and University of Bristol Sakir Sezer, Queen’s University Belfast 1 Introduction: Rationale/Justification Industrial Control Systems (ICS), composed of combinations of hardware, software and ICT networks, orchestrate the myriad of functions needed to execute complex tasks such as the delivery of utility services and the operation of intricate and disparate manufacturing processes. ICS are examples of cyber-physical systems – digital systems that affect and are affected by, physical processes – whose use is growing through developments in smart-city technologies and the rapid emergence of the Internet of Things. Such systems are increasing in importance as techno-social components of the Critical National Infrastructure (CNI) of the future and as they extend their scope, becoming ubiquitous, accessible and transformative to wider society and the economy, the need to understand their security characteristics also increases. To date the Research Institute in Trustworthy Industrial Control Systems (RITICS) activity has focussed on identifying existing technical and practical problems that surround the development of secure and trustworthy ICS. In order to develop realisable solutions to these problems RITICS has conducted a research programme that includes work in: • Theory and analysis • Simulation and experimentation • Testing and implementation To effectively execute this mission, the need for a simulation/lab space where components and instances of investigated digital systems may be physically configured for 'close to real-world' fidelity is vital. RITICS partners have developed small-scale testbed facilities. This white paper surveys the current range of facilities, summarises the lessons learnt, presents the issues with linking these facilities and concludes with a forward look. Our ambition is to interconnect the existing systems together in order to achieve the scale of real- world systems and to use the capabilities to accelerate and increase efficiency/effectiveness of the UK investment. This will enable us to • better understand the interdependencies between different sectors • better understand the similarities and differences between Information Technology (IT) and Operational Technology (OT) • test and prepare for targeted and untargeted attacks • provide training to close the skills gap • validate various theories about how to deal with new and unknown threats • extend understanding of system-user relationships across an array of sectors
Transcript of Open Testbeds deliverable-final - RITICSritics.org/wp-content/uploads/2018/07/Open... · PETRAS The...
OpenTestbedsforCNI
AuthorsChrisHankin(editor),ImperialCollegeLondon
DeephChana,ImperialCollegeLondonBenGreen,LancasterUniversityRafiullahKhan,Queen’sUniversityBelfastPeterM3,NationalCyberSecurityCentrePeterPopov,CityUniversityofLondonAwaisRashid,LancasterUniversityandUniversityofBristolSakirSezer,Queen’sUniversityBelfast
1Introduction:Rationale/JustificationIndustrialControlSystems(ICS),composedofcombinationsofhardware,softwareandICTnetworks,orchestratethemyriadoffunctionsneededtoexecutecomplextaskssuchasthedeliveryofutilityservicesandtheoperationofintricateanddisparatemanufacturingprocesses.ICSareexamplesofcyber-physicalsystems–digitalsystemsthataffectandareaffectedby,physicalprocesses–whoseuseisgrowingthroughdevelopmentsinsmart-citytechnologiesandtherapidemergenceoftheInternetofThings.Suchsystemsareincreasinginimportanceastechno-socialcomponentsoftheCriticalNationalInfrastructure(CNI)ofthefutureandastheyextendtheirscope,becomingubiquitous,accessibleandtransformativetowidersocietyandtheeconomy,theneedtounderstandtheirsecuritycharacteristicsalsoincreases.TodatetheResearchInstituteinTrustworthyIndustrialControlSystems(RITICS)activityhasfocussedonidentifyingexistingtechnicalandpracticalproblemsthatsurroundthedevelopmentofsecureandtrustworthyICS.InordertodeveloprealisablesolutionstotheseproblemsRITICShasconductedaresearchprogrammethatincludesworkin:
• Theoryandanalysis• Simulationandexperimentation• Testingandimplementation
Toeffectivelyexecutethismission,theneedforasimulation/labspacewherecomponentsandinstancesofinvestigateddigitalsystemsmaybephysicallyconfiguredfor'closetoreal-world'fidelityisvital.RITICSpartnershavedevelopedsmall-scaletestbedfacilities.Thiswhitepapersurveysthecurrentrangeoffacilities,summarisesthelessonslearnt,presentstheissueswithlinkingthesefacilitiesandconcludeswithaforwardlook.Ourambitionistointerconnecttheexistingsystemstogetherinordertoachievethescaleofreal-worldsystemsandtousethecapabilitiestoaccelerateandincreaseefficiency/effectivenessoftheUKinvestment.Thiswillenableusto
• betterunderstandtheinterdependenciesbetweendifferentsectors• betterunderstandthesimilaritiesanddifferencesbetweenInformationTechnology(IT)and
OperationalTechnology(OT)• testandpreparefortargetedanduntargetedattacks• providetrainingtoclosetheskillsgap• validatevarioustheoriesabouthowtodealwithnewandunknownthreats• extendunderstandingofsystem-userrelationshipsacrossanarrayofsectors
TherewillalsobetheneedtodevelopabusinessmodelforhowtheOpenTestbedsmightoperate.
2.TheUKLandscape
TheLancasterICSTestbedThemostextensivetestbedfacilitiesdevelopedwithinRITICShavebeendevelopedbytheMUMBAprojectattheUniversityofLancaster.Inadditiontotheirlab-basedtestbedwhichcanbeconfiguredinanumberofways,theyhavealsodevelopedatable-topwatertreatmentdemonstrator.
Ahigh-levelviewofLancaster’sICStestbedisshowninFig.1[1].ThearchitectureisbasedonthePurdueReferenceArchitecture.CurrentlysplitacrosssixManufacturingZones,anICSDemilitarisedZone,andanEnterpriseZone(withitsownseparateDemilitarisedZone),allequipmentinthetestbedisphysical(unlessotherwisenotedasVirtualisationPlatforminFig.1).ItisimportanttonotethatLancaster’stestbedhasfocusedonthedevelopmentofsystemsanddevicesacrossLevels0,1,2,3,DMZand4ofthePurduemodel.
Figure1:NetworkDiagramofSecurityLancaster’sICSTestbed
ITRC’sDAFNIprojectTheInfrastructureTransitionsResearchConsortium(ITRC)isaconsortiumof7universities(Cambridge,Cardiff,Leeds,Newcastle,Oxford,SouthamptonandSussex),investigatingwaystoimprovetheperformanceofinfrastructuresystemsintheUKandaroundtheworld.Theirresearchishelpingbusinessesandpolicymakerstoexploretheriskofinfrastructurefailureandthelong-termbenefitsofinvestmentsandpoliciestoimproveinfrastructuresystems.
TheDataandAnalyticsforNationalInfrastructure(DAFNI)projectwillcreateanationalinfrastructuredatabaseforvisualisationandanalysis.Itwillbeashared,securesystemforacademicresearchandaresourceforbusinesses,innovatorsandpolicy-makers.
DAFNIwelcomesideasabouthowitwillworkwithdifferentpartners,andonissuessuchas:security,access,suitablebusinessmodels;andnextsteps.
AkeyfeaturewillbeDAFNI’ssimulationandvisualisationfacilitiestoallowuseofmodelsinamoreflexibleway,enablingthesystemsofsystemsanalysisandincorporatingobservedandsimulateddatasets.DAFNIwillbenefitfromtheexperienceoftheITRC)whichhasbeendevelopingaone-stopdatabaseforUKinfrastructure(NationalInfrastructureSystemsMODel–NISMOD).It’smuchmorethanacurationofdata,andallowsrepresentationofinterdependenciestoinformplanningdecisions,includingviaavisualisationdashboard.AlthoughNISMODcontainsover400datalayers(representingmultiplesectors,demographics,economics),theinfrastructuresectorneedsgreaterdetail,torepresentindividualbuildingsandtodevelopplausibleconnectivitynetworks,whichDAFNIcandeliver.ITRC-Mistralisdevelopingameta-databasetogiveuserstheexperienceofasingleinterface,althoughitbringstogethermanydatabases,andthisisthemodelthatwillbeappliedtoDAFNI.
OnechallengewillbehowtomakeDAFNIsuccessfuloperationally.DAFNI’svisionistobuildanenvironmentwherepeoplecantrydifferentsolutions,whichmeansbeingresponsivetoallusers.ExistingmodelsmightincludeJASMINthatusesthedesktopasaservicetoolusingastandardtoolkit,withnorestrictionsonusers.
5GTestbedsTheDepartmentofDigital,Culture,MediaandSport(DCMS)areinvestingina5GtechnologytestnetworkaimingtoputBritainattheforefrontofthenextwaveofmobiletechnology.
5GresearchinstitutionsatKing’sCollegeLondonandtheUniversitiesofSurreyandBristol,havebeenawarded£16mtodevelopthecutting-edge5Gtestnetworkwhichwillbringacademiaandcommercialcompaniestogethertotrialthetechnologyandmakesurepeopleandbusinessescanrealisethebenefitssooner.
Thistestnetworkwilltrialanddemonstratethenextgenerationofmobiletechnologyandisthefirstpartofafour-yearprogrammeofinvestmentandcollaborationintheGovernment’snew5GTestbedsandTrialsprogramme.
Theuniversitieswillworktogethertocreatethreesmall-scalemobilenetworkswhichtogetherwillformthetestnetwork.Eachnetworkwillhaveanumberoftheelementsexpectedinacommercial5Gnetwork-includingmobilesignalreceiversandtransmittersandthetechnologytohandle5Gsignals-tosupporttrialsofitsmanypotentialuses.
Otheracademicinstitutions,industryandlocalauthoritieswillalsobeabletobidforfurtherfundingtobepartofthisprogrammefrom2018/19onwards.Furtherdetailsonopportunitiesandthefundingavailablearepublishedintheprospectus.
UKCRICTheUKCollaboratoriumforResearchinInfrastructure&Cities(UKCRIC)willprovideleadershipandsupportforthedevelopmentandgrowthofacoordinatedandcoherent,worldclass,UK-basednationalinfrastructureresearchcommunity,spanningatleast14universities.Itwillengagegovernment,cityandcommercialpolicymakers,investors,citizensandacademiainajointventurethatdrivesinnovationandvaluecreationintheexploitationofservicesprovidedbynationalinfrastructure.Throughcentralcoordination,providingafocalpointforknowledgetransfer,UKCRICwillsupportastep-changeinthenation’sapproachtoinfrastructureinvestment.Itwillalsodevelopacommercialresourcethathasconsiderableexportpotentialforaninternationalmarketthatisvaluedat$57trillionintheperiodupto2030.
UKCRICwillunderstandhowtomakethesystemofsystemsthatconstitutesthenation’sinfrastructuremoreresilienttoextremeeventsandmoreadaptabletochangingcircumstancesandcontexts,andhowitcanprovideservicesthataremoreaffordable,accessibleandusabletothewholepopulation.
PETRASThePETRASHubhasfundingforthecreationofanumberofdemonstrators.Theprojectisstilldebatingwhatformtheseshouldtake.
UniversityofBristolWiththeMumbaprojectteam’smovetotheUniversityofBristol,anewICStestbedisbeingsetupthatwillincludemultiplefieldsitesandindustrialprocessestosupportresearchonsecurityofindustrialcontrolsystems,includingbothlegacyandnon-legacydevicesandIndustrialInternetofThings(IIoT).
3.InternationalFacilitiesHolmetal[3]presentanoverviewofinternationalfacilitiesasattheendof2015.Theyidentify30testbedsthatwereeitherplannedorinoperation,almosthalfofwhichwereintheUS.TheyciteSiaterlisetal[4]whopresentthefollowingcriteriathatcybersecuritytestbedsshouldfulfil:
• Fidelity:tobeasaccuratearepresentationoftherealsystemaspossible• Repeatability:repeatedrunsshouldgiveconsistentresults• Measurementaccuracy:observingrunsshouldnotperturbtheoutcome• Safeexecutionoftests:theeffectofatestshouldbecontainedwithinthetestbed
Thesearereasonablerequirementstoexpectofanytestbedfacility.
The iTrust Water testbeds (Singapore) are small-scale networks within a controlled laboratoryenvironment,composedofasmall-scalewaterdistributionnetwork (WADI)anda treatmentplant(SWaT). The testbeds are used for security analysis for water distribution networks, to assessdetectionmechanismsforcyberandphysicalattacks,aswellastounderstandcascadingeffectstootherconnectedsystems.The[iTrust]InternetofThingsAutomaticSecurityTestbed(Singapore)isasmall-scale laboratory composed of GPS simulator, Wi-Fi localization simulator, time simulator,movementsensor,tosimulatethedifferentenvironmentalconditionsinwhichIoTdevicesoperate.Thetestbedsupportsstandardandcontext-basedsecuritytestingandanalysisforIoTdevicesunderrealconditionsagainstasetofsecurityrequirements.Power-Cyber(USA)isasmartgridtestbedwiththepurposetoperformvulnerabilityassessment(i.einspect weaknesses within the infrastructure), design mitigation methods, and develop cyber-physical metrics (i.e metrics combining cyber-physical properties), cyber forensics tools (exploreways to detect cyber-attacks specific to industry protocols and field devices), and securemodels(explorationofinnovativesecurityapproaches).TheUniversity of Illinois atUrbanaChampaign has developed the Cyber-Physical ExperimentationEnvironmentforRemoteAccessDistributedICS(CEER).Asummaryofthiseffort(includedverbatimhere) has been extracted from: https://iti.illinois.edu/research/energy-systems/cyber-physical-experimentation-environment-radics-ceer
“Thegoalofthisprojectistoprovideatestbedonwhichprospectivetechniquesandtoolscanbedeveloped,refined,andvalidatedinacontextwithunprecedentedsystemfidelity.Weareclosingthegapbetweenneedsandstateoftheartthroughatestbed,CEER,thatisinnovativeinseveralways.
CEERbringstotheICSdomainforthefirsttimeproductionqualitysoftwaretoflexibly(andremotely)defineexperiments,configuretestbedresources,andrunexperiments.Itbringsthefruitsofstate-of-the-artmodelingofgridsystemstoprovidesyntheticbutrealisticdynamicgridstate.Itbringscutting-edgeappliedresearchintemporalcoordinationofrealdevices,deviceemulation,andsimulatorsofdiversekindstoenablecreationofexperimentaltopologiesthataremuchlargerthantheensembleofphysicalICSdevicesinthetestbed.CEERbringsbest-of-breedICSsysteminstrumentationandmonitoringtechnologytoenableuserstocloselytracktheresultsoftesting.Itwillbeabletoaccuratelyrepresentthesmartgridinteractionsfromgeneration,transmission,anddistribution.Itwillalsosupporthigh-fidelityexplorationofassetsineachofthesedomains,including,butnotlimitedto,generationassets,gridcomponentsintransmissionanddistributionsubstations,controlcenteroperation,andadvancedmeteringinfrastructure.”
Anapproachtakenbythecolleaguesbehindthistestbedistousehigh-fidelitysimulatorsofthe“physicalworld”,whichallowsforclose-to-trueimpactofcyber-attackstobeaccountedfor.
TheUSNationalInstituteofStandardsandTechnology(NIST)isdevelopingacybersecuritytestbed(seeFig.2[2]).
Figure2:TheNISTTestbed
Theaimistomeasuretheeffectofprevailingstandardsandguidanceontheperformanceofcontrolsystems.Thetestbedisdesignedasaseriesofenclavesthataddressdifferentindustrialsectors.ThetestbedusessimulationwhereappropriatewithHardware-in-the-Loop(HIL)componentssimulatingtheinterfacesbetweensensors/actuatorsandthecontroller.Thedifferentenclavesallowstudyofcontinuousprocesses(suchaschemicalmanufacture),discreteprocesses(suchasautomotiveassembly)andhybridprocesses(suchaspharmaceuticalmanufacture).Performanceismeasuredusingappropriatetechnicalperformanceindicatorsfortheprocesses.
TheDepartmentforBusiness,EnergyandIndustrialStrategyCivilNuclearTeamisintheplanningphaseregardinganupcomingjointexercisewithEstonianMODofficials.TheEstonianofferisanetworkdefenceexercisescenariomakinguseofafully-equippedcybertestrangeinTallinn,andwillinvolveparticipantsfromacrossthecivilnuclearsector.TheNationalCyberSecurityCentrewillalsohaveinvolvementintheexerciseusingtheEstonianDefenceForcesCyberRange,consistingofasimulatedofficenetworkconsistingoftypicalserversandworkstationsaswellasfacilitiessupportsystemswillbeimplemented.
HitachiaredevelopingaSecurityTrainingArena(SeTA)attheirOmikaWorksinJapan.Theemphasisofthiscentreistotrainoperatorshowtodealwithcyberincidentsinanuclearpowerplant.TheyplantorunjointexerciseswiththeUK(andpossiblyUS)in2018.TheyhavehadpreliminarydiscussionswithImperialCollegeLondonandRoyalHollowayUniversityofLondonaspotentialacademicpartnersinthisprogramme.
ThelargestsecurityclusterinEuropeissituatedinTheHagueSecurityDelta.In2015,theypublishedaproposalforanational,multi-sectortestbed[5].Atthetimeofwriting,theyarestillrecruitingpartnerstoassistintheconstructionofthefacility;theinitiativeissupportedbyTNO,KPNNVandtheMunicipalityofTheHague.TheminimumrequirementsfortheDutchnationaltestbedareasfollows:
• Theplatformshouldhosttestlabsformultiple,differentcriticalinfrastructuresectors• Theplatformshouldgenerateknowledgethatcanbeusedtocreatesolutionsforcritical
infrastructureequipment• Itshouldbeavailablefortrainingofinformationsecuritystaffonthreatsandexploits• Thetestbedshouldfacilitatethecreationofanetworkofhighlyqualifiedinformation
securitystaff• Thetestbedfacilityshouldperiodicallyproduceconfidentialreportsaboutnewlydiscovered
threatsandvulnerabilities• Thetestbedfacilityshouldprovideopenandfreelyavailablesecurityreportswiththe
securitysolution• Thetestbedfacilityshouldturnsecurityrequirementsintonewindustrystandards• Thetestbedfacilityshouldeducatecriticalinfrastructurecompaniesinbestpracticesand
lessonslearnedfromacrossallsectors• Thetestbedfacilityshouldestablishcooperationandinformationsharingamong
participatingpartners
TheserequirementsoverlapsignificantlywiththeambitionthatweoutlinedinSection1above.
4.DesignIssuesandLessonsLearntThekeydesignissuesandlessonslearntfromtheconstructionoftheLancastertestbed[1],whichalsofindechoesintheothercitedpapersare:
1. Theneedtoinclude,eitherphysicallyorvirtually,adiverserangeofdifferentdevices(vendorsandversions)
2. Theneedforscaletoprovidefaithfulrepresentationsofrealsystems3. Appropriatemechanismstomanagethecomplexityoftheinfrastructure
DiversityAneffectivetestbedshouldbeabletomimicavarietyofICSsetups.Keyquestionsinclude:
1. Selectionofdevicesandprotocolsforinclusion;2. Providingdifferentconfigurationsofdevices/manufacturerstypicalinICSsettings;and3. Balancingdeviceandprotocoldiversityagainstotherrequirements,suchasthe
implementationofthephysicalprocessitself.
ExperienceswithintheLancastertestbedhavehighlightedthat[1]:
• Deviceandtechnologyselectionsshouldbemarket-driven;• Fieldsitesinatestbedshouldrepresentdifferentreal-worldscenariossuchashomogeneity
andheterogeneityofvendorsaswellascombinationsoflegacyandnon-legacydevices;• Processdiversitycanhelpmodelstealthattacksthatexploitphysicalaspectsoftheprocess
butthatsuchprocessdiversitymaybetraded-offinfavourofdiversityofdevicesandfieldsites.
ScaleSoftwaredoesnotprovidesimulationsofmanyessentialtypesofdevices,i.e.fromdifferentvendorsorthesamevendorbutdistinctiveversions.Theaccuracyandreliabilityofsuchsimulationsinmimickingreal-lifeoperationsalsoremainanissue.Therefore,whilethecostofphysicalequipmentcanbealimitingfactor,thebenefitsitcanbringinrelationtoexperimentalrigourisanoverridingconstraint.Ontheotherhand,virtualisationandVLANscanprovideeaseofintegrationandscalingofthetestbedinfrastructure[1].
ComplexityAlthoughtheunderlyingarchitecturemaybecomplexandinvolveanumberofnetworkzones,thisshouldbeastransparenttotheuseraspossible.Transparencycanbeachievedbyprovidingasinglepointthroughwhichaccesstoandextractionofdatafromthedifferentzonescanbemanaged.AsecondlessonlearntbytheLancasterteam[1]isthenecessitytocreateandmaintaingooddocumentationofthetestbedasitevolves.
FurtherLessonsTheNSFreportonCybersecurityExperimentationoftheFuture[6]providesadetailedroadmapforthedevelopmentoffutureexperimentationinfrastructureoverthenear-term(3years),mid-term(5years)andlong-term(10years).Thereportalsoreviewstheexperienceof46US-basedexperimentalfacilities.
Thetop5recommendationsfromthereportareasfollows:
• Focussingonmultidisciplinaryexperimentationdrawingonboth“hard”sciencesandsocialsciencewillhavethegreatestimpactinacceleratingcybersecurityexperimentationinthenearterm.
• Theabilitytoaccuratelyrepresentfullyreactionarycomplexhumanandgroupactivityinexperimentswillbeinstrumentalinenvironmentsthatrealisticallyrepresentreal-worldsystems.
• Creatingopenstandardsandinterfacesisamid-termpriority.• Researchanddevelopmentusingthelatestadvancesindatascienceisneededtocreate
reusable,extensible,validatedexperimentdesigns.• Researchinfrastructuremustbeusablebyabroadrangeofresearchersandexperts,notjust
restrictedtocomputerscienceresearchers.
5.LinkingTestbedsMostUKacademicinstitutionsandresearchcentresareprovidedconnectivitybyJanet,ahigh-speed,secureandreliableworld-classnetwork.Janetprovidesatleasta10Gb/sphysicallinkandaClass-BIPaddresspool,enablingalltypesofInternetserviceswithinUKacademiccampuses,includinglow-latencyVoiceoverIP(VoIP).However,experimentallabfacilitiesandresearchtestnetworksaresignificantlyconstrainedintakingadvantageoftheJanetinfrastructure.InordertoavoidsecurityandQualityofService(QoS)relatedthreatstotheJanetnetwork,researchlabfacilitiesare,inmostcases,disconnectedandrelyonexternalmulti-megabitADSLlinesvialocalISPproviders.Theselimitationsnotonlyconstrainresearchcapabilitieswithintheseinstitutions,butalsoimpairnationalandinternationalcollaborationsthatrequirehigh-speedconnectivityamongstcollaboratingpartners.
Anotherkeyfactorthatlimitstheresearchcapabilities,qualityandeffectivenessisthelimitedavailabilityofresourceswithinacademicinstitutionsandresearchcentres.SettingupanexperimentallabfacilityorresearchtestnetworkisextremelyexpensiveespeciallyintheICSandSCADAdomainduetotheneedforexpensivedevices/equipment.Thenon-availabilityofstate-of-the-artexperimentalresourcessignificantlylimitsresearchpotentialofindividualacademicinstitutions.
Theaimforlinkingtestbedsistoenableallpartnerinstitutionswithleadingedgeresearchcapabilities,experimentallabfacilitiesandtestnetworksbysharingresourcesoversecureandreliablehigh-speedJanetinfrastructure.TheteamatQUBhaverecentlyproposedanapproachwhichisuniqueofitskindbyinter-linkinglabfacilitiesofalluniversitiesacrosstheUKasshowninFigure3.ThisproposaldoesnotjustfocusonlinkingICSandSCADAfacilitiesbutproposesamoregeneralnetworkoftestbeds.
AshighlightedontherightinFigure3,thesharingnaturewillenableallparticipatinginstitutionstobenefitfromthesamesearchfacilitiesandhaveaccesstotest/experimentalnetworkswhichtheywerelackingindividually.TheproposedresearchnetworkwillbebuiltuponJanet’snetworkinfrastructureusingconfigurablemulti-gigabitVPNtunnels,providingconnectivityofupto10Gb/samongstthepartners,whilefacilitatingstrictisolationfromeachnode’smaincampusnetwork;asimilararchitectureisalreadyunderevaluationtoallowexternalconnectivitytotheLancastertestbed.Centralisednetworkadministrationandmanagementwillprovideprojectspecificconfigurationofthenetwork(topology,bandwidth)andexternalconnectivitytonationalandinternationalpartners,andtheInternet,viaasecuregatewayusingJanetandthird-partyISPs.The
baselinearchitecturewillbelaidoutassuchthatthenetworkcanbescaledtoexpandbeyondthecurrentpartners,capableofservicingtheUKacademicresearchcommunityformanyyearsahead.
Figure3:AProposedUKNetwork
DirectBenefitstoUKUniversitiesandResearchCentresTheacademicandindustrialpartnerswillbeabletotakeimmediateandfulladvantageoftheproposedconnectivitybyprovidingthem:(i)anextendedresearchnetworkinfrastructureforexperimentalstudies,(b)accesstothecollectivepoolofstate-of-the-artexpensivetechnologies,tools,testbedsanddatasets,whicharecurrentlyonlyavailabletotheowninginstitution,(c)providethenecessaryinfrastructureforgeneratingresearchspecifictrafficsamples,logfilesandotherspecialistdatasets,andthefacilitytosharealargerepertoireofexistingdatasetsamongstthepartners,(d)limitedaccesstospecialistnetworkingskillsanddomainknowledgebytakingadvantageofthenetworkanditsdedicatedstaff.
Thescaleanddiversityoftheproposedresearchnetworkandtestfacilitiesprovidesuniqueresearchopportunitiesforendusers,suchasenterprisesandcorporationsrelyingonlargeITnetworksandITsecurity.Theproposedresearchnetworkwillprovidenumerousbenefitsforpartnersbysharingorprovidingaccesstoexpensiveandrareresources,accesstomorerealisticexperimentalenvironmentandimprovingresearchcollaboration.
QUBhasanextensiveexperimentalnetworkandtestlabinfrastructurecloselycoupledtoasystempenetrationtestandtraininglab.Theinter-linkedresearchnetworkwillextendthattestcapabilityandprovideamorerealisticanddistributedICSsystemtoexperimentwith.Inadditiontothe
Aberdeen
Dundee
EdinburghGlasgow
Belfast
Liverpool Manchester
Leeds
Sheffield
LeicesterAberystwyth
Birmingham
Cambridge
Cardiff London
Southampton
Plymouth
Bristol
Oxford
Canterbury
Lancaster
Layer3VPNTunnelLayer2LightPath
janetQUB
UoE
USW
UE
DMU
UB
Internet
ISPJanet
Gateway
Network AdministrationManagement and SecurityTraffic monitoring and acquisition
RHUL
Lanc.Bris.
benefitsofscalethatwillaccruetoexistingRITICStestbeds,suchastheoneatLancaster,thenetworkwillalsobenefitothernewRITICSpartners.Forexample,theplannedtestbedatBristolwillbelinkedintothenetworkaswellasthoseuniversityfacilitiescurrentlysupportedbyAirbus.ThecollaborativeprojectbetweentheUniversityofSouthWales(USW)andAirbusDefenceandSpacecalledSCADACyberSecurityLifecycle(SCADA-CLS),istargetingthedevelopmentofacyberforensiccapabilityforSCADAprocesscontrolsystems.Theinter-linkedresearchnetworkwillprovideanextendedICSnetworkthatUSWcaneffectivelyutiliseforforensic/incidentmanagementtriageprocessmodelling,andthedevelopmentofSCADAforensictoolsfordataacquisition,incidentmanagementandsituationalawareness,usingSCADAtestfacilitiesatQUBanditsFP7/H2020partners.DeMontfortUniversity’s(DMU)CYRANcyberrangetechnology,whichwillbedirectlyaccessiblebyallpartners,providesaplatformforcyber-attack/defencescenariosforexperimentalresearchandforeducationalgamesthatincludephysicalartefactssuchasPLCcontrolledproductionlinesandfiltrationsystems.DMUwillbeabletoextendtheCYRANcapability,accessingPLCcontrollersatQUB,USWandEuropeanpartners.SCADAandothertypesofICSrelatedlargedatasetscannowbegenerated,takingadvantageoftheadditionalphysicalresourcesfromthepartners’testbeds.ThisdataisdirectlyrelevanttoexistingprojectswithinDMUonSCADAForensics,undertakenwithAirbusGroupInnovationsaswellasresearchonPrivacyMetricsandIncidentResponsemanagement.
Bycombiningmanytestnetworkswithuniqueproperties,moregeneralcybersecurityresearchprojectswillalsobenefit.Logfilesfromnextgenerationfirewalls(ngFW)withintheproposedresearchnetworkwillbeusedtoanalysemalicioustrafficinLANnetworks;workingwithmultiplengFWdatawillenabletheanalysisacrossawideareanetwork.Akeybenefitisthegenerationoflargelogfileswithintheexperimentalnetworkwithoutbeingconstrainedbytheprivacyandethicalchallengesoflivecampusnetworks.Dedicatedmonitoringandinterceptiontechnologywithintheproposedresearchnetworkwillprovideadvancedtrafficvisibilityandpacketprocessingcapabilityformanyprojects.Theproposedresearchnetworkwillallowpartners(a)tofurtheranalyserepetitiveexternalattackstotheirITinfrastructurebyreplayingattackpatterns,(b)usecross-sitetestcapabilitiestoundertakestressandpenetrationtestingonneworexperimentalsecurityandnetworkappliances,and(c)assessnewcybersecurityarchitecturesandthreatmitigationstrategiesoncorporatenetworksandwebsites.
Datasetandtesttrafficgenerationandsharingisoneofthemostimportantandchallengingtopicsinnetworkandcybersecurity.Availabledatasetssuchasinterceptedtrafficareconstrainedandinmostcasesrelevanttoaspecifictypeofthreat.Privacyandethicalconsiderationspreventtheuseofanyintercepteddata,suchasfromaUniversitycampusnetwork.Furtherconstraintsarethatmalware,APTandDDoSrelatedprojectsrequiresfreshdatasetsandtrafficcontainingtargetedthreatsinordertounderstandtrafficpatternsrelatedtothreats,andforoptimisingdetectionalgorithmssuchasmachinelearningclassifiers.Theproposedresearchnetworkbringstogetherhighlydiversetestnetworksatascaleandthetrafficcapacityofalargenetwork,providingauniqueopportunityforgeneratingtailoreddatasetsandsampletraffic.
Transportexperimentallabfacilitiesarequiteexpensivetoestablishandonlyfewuniversitieshaveadvancedtestbeds.Theinterconnectedresearchnetworkwillbeofsignificanceforimprovingcollaborationamongstacademicinstitutionsandeffectivelysharingtheirtransportlabfacilities.BirminghamCentreforRailwayResearchandEducation(BCRRE)oftheUniversityofBirmingham(UoB)hassignificantexperimentallabfacilitiesforresearchinaddressinggrandsystem-wideaswellascomponentlevelchallenges.UoBrailwayresearchcoversvariousaspectsincludingsafety,operationsandmanagement,dataintegrationandcybersecurity.Recently,theUKRailResearch
andInnovationNetwork(UKRRIN)researchcentrehasbeenestablishedforsupportingnewinnovationsinrailtransport.UKRRINaimistobringtogetherexistingfacilitiesatdifferentacademicinstitutionsandaccelerateinnovationandnewproductdevelopmentintherailindustry.AspartofUKRRIN,UoBwillcarryoutresearchindigitalrailsystemscoveringcybersecurity,trafficmanagementandrailwayconditionmonitoringandsensing.UniversityofNewcastle(UoN),LoughboroughUniversity(LU)andUniversityofHuddersfield(UoH)withinUKRRINwillcollaborateonhighvaluerollingstocksystems,assetoptimisationandthrough-lifemanagementandenergymanagement.Whereas,UniversityofSouthampton(UoSA),UniversityofSheffield(UoS),UniversityofNottingham(UN)andHeriot-WattUniversity(HWU)arecarryingoutresearchonrailwayinfrastructurewithinUKRRIN.TheproposedinterconnectedresearchnetworkwillbethemediumenablingallpartnersofUKRRINtocollaborateeffectivelyandshareexperimentalresources.
ProposedSystemArchitectureTheproposedinter-linkedresearchnetworkwouldbedevelopedinmultiplephases,takingadvantageoftheavailableJanetconnectivityandsparebandwidthcapacityoftheacademicinstitutes.
Figure4outlinestheoverallnetworkandtestbedarchitectureamongstthepartners.Aninitialphasewouldtargetthedevelopmentofthebasicoverlayarchitectureontopoftheexistinglayer3Janetconnectionviamulti-gigabitVPNtunnelsandtheestablishmentofthenetworkwiththenecessarynetworkadministrationandmanagementtoolsandsupportresources.Acontrolcentrewithnetworkadministrationandmanagementtoolswouldbeestablished,responsiblefortheadministrationandmanagementofthelinksamongstthepartners,JanetandtheexternalconnectivitytotheInternet.Phase-2developmentwouldprovideadditionalphysicallinkcapacitiesandexternalconnectivitytointernationalandindustrialpartners.
Figure4:TheProposedArchitecture
Janetlayer2LightPathprovidesthelowestcostandhighest-bandwidthconnectivity,fortheproposednetworkitalsointroducessignificantchallengesprovidingthenecessarylayer3networkinfrastructureandsupportingcybersecurity(malware,DDoS,cloud),IoT,andIndustrialcontrol
relatedtestlabs.NumerousSDNandcloudnetworktestbedscantakefulladvantageofthelayer2bandwidthsuchasstreamingterabytesofdatabetweenbig-datalabs.
RequirementsforSecureInter-linkingofDiverseTestbedsTheobjectiveistodevelopaplatform,basedonasecureoverlaynetworkarchitecture,forinterconnectingvariousacademicandindustrialtestbedsintoalargerUKwideresearchnetwork.AsdepictedinFigure4,suchaprivateoverlaynetworkapproachhasthreebasicrequirements:(i)networkconnectivitykit,(ii)centralizedoperationsandmanagementand(iii)highspeedJanetnetwork.
NetworkConnectivityKitThenetworkconnectivitykitenablesremotetestbedsitesfrompartnerstobeconnectedtotheprivateoverlaynetworkasshowninFigure4.Thesolutionisscalableandnewtestbedscanbeeasilyintegratedwithinthenetworkwithoutmajortechnicalsupport.Tobecomepartoftheinterconnectedphysicaltestbedinfrastructure,eachacademicorindustrialpartnershouldbeprovidedwithanetworkconnectivitykitorrack-mountedkit.Thebasicarchitectureoftherack-mountedkitisshowninFigure5andconsistsof:
• AFirewall/Router/VPNwhichwillbemanagedfromtheCSITTestNetworkManagementcentre.
• Adistributionswitchthathasportmirroringcapabilitiestopermittrafficcapture.• Trafficdatastoragecapability.Terabytesofnetworktrafficdatamayneedtobecaptured
andstoredforlateranalytics.
Figure5:ArchitectureoftheNetworkConnectivityKit
ThenetworkconnectivitykithasaVPNclientandappropriatelydimensionedcommunicationandstoragehardware.ItistailoredfortheproposedresearchnetworkcomprisedofappliancesforatraditionalIPnetworkandanSDNnetwork,capableofsupportingadvancedfirewall,andVPNtunnelwithVLANsegregationcapability.
OperationsandManagementTheproposedresearchnetworkhascentralizedmanagementformanagingconnectivitybetweendistributedtestbedsites,networkaccesscontrolanddataacquisition.Italsohasadata-set(sampleddataandtrafficpatterns)repositorywithpostprocessing,indexingandaccesscontrol.Aneffective
managementandadministrationstructureisessentialtoensurethesuccessoftheproposedresearchnetworkanditsefficientutilisationbythepartnersandthewiderresearchcommunity.
ThecentralizedoperationsandmanagementsystemwillbehostedattheCentreforSecureInformationTechnologies(CSIT)datacentre.TheproposedresearchnetworkwillbemanagedbyQueen’sUniversityBelfast,aspartofCSIToperationandmanagementinfrastructure,incollaborationwithallpartners.
HighspeednetworkThehighspeedlinkcapacitieswillbeleasedfromJanetwhichwillactasthebackboneandfabricoftheproposedresearchnetwork.ThemajorityofcostinthedevelopmentofproposedresearchnetworkisassociatedwiththeleasingofcommunicationlinksfromJanetandprovidingthenetworkconnectivitykittoeachpartner.GoingbeyondthecurrentQUBproposal,itshouldalsobeconsideredhow4Gtechnologies,asusedatLancaster,canbeincorporatedintothenetworkinasecure,reliable,andmanagedformat.TherecouldbeafurtherextensiontowardsPSTN/GSMservices,inwhichlegacydial-uptechnologiesmayalsobeapplied.
UseCaseExamplesInter-linkingexperimentalresourcesfromacademicandindustrialpartnersmakestheproposedresearchnetworkquiteheterogeneousconsistingofdiversetestbedsinalldifferentresearchareas.Basedontheresearchtopic,apartnercanrequestresourcesinaspecificdomainfromcontroloperationsandmanagementcentre.TheControlcentrewillcreateasecuresegregatedVLANwithdedicatedexperimentalresourcesbasedontherequest.Theallocatedresourcescanthenbeexploitedbythepartnerstoexperimentanddetermineeffectivenessoftheirdevelopedtechnologiesandresearchtools.Toillustratetheutilityofthenetwork,thissectionpresentstwoICSusecaseswheretheproposedresearchnetworkcanbeutilized.
DistributedIntrusionDetectionandPreventionAnIntrusionDetectionSystem(IDS)monitorsanetwork/systemformaliciousactivitiesorviolationofpoliciesandraisesalerts.Whereas,IntrusionPreventionSystem(IPS)complementsIDSbyalsotakingdefensiveactionswhenamaliciousactivityisdetected.SeveralacademicinstitutesandresearchcentresareactivityinvolvedinIDS/IPSresearchtoimprovedetectionefficiencyandeffectivelyhandleemergingthreats.NISTpublishedrecommendationsthatIDS/IPSsystemsshouldbehybrid,distributedinnature,havedecentralizeddecisionmakingandcentralizedmanagementandrefinementofdetectedevents.ThehybridIDS/IPSsystemsperformbothhost-basedmonitoringaswellasnetwork-basedmonitoringformaliciousactivitiesdetection.Thedistributednaturesuggestsmultiplesensorstobedeployedinsysteminsteadofrelyingonasinglesensorforredundancyandbettermaliciousactivitiesdetection.
SeveralICSsystemsaredistributedinnaturee.g.,powersystems.ToinvestigateIDS/IPStechnologiesforadistributedICSnetwork,apartnercanrequestresourcesfromthecontrolcentreofproposedinterconnectedresearchnetwork.ThepartnerwillbenefitfromnothavingitsownbututilizingsharedICStestbedsfromotherpartnerinstitutes.Thiswillenablethepartnertocontinueresearchinthistopicevenifitislackingequipment.
Figure6:TheDistributedIDSscenario
RealisticExperimentalPowerSystemsPlatformTheproposedresearchnetworkcanbeusedtoconductresearchinamorerealisticdistributedexperimentalplatform.E.g.,powersystemsarehighlydistributednowadaysduetodevelopmentofrenewableandgreenenergysources(e.g.,windfarms,solarpanels,etc).Thistrendisbecomingmoreandmorecommonandgreenelectricitysourcesarepredominantlylocatedatgeographicallyisolatedareas.Severaluniversitiesareconductingresearchondistributedgenerationandtransmission,microgridsandsubstationsincludingQueen'sUniversityBelfast,ManchesterUniversityandStrathclydeUniversity.Distributedgenerationandintegrationintomaingridtakesbenefitfromsynchrophasortechnology.SynchrophasortechnologyincludesacontrolcentrethatreceivesGPStimestampedelectricalmeasurementsfrommicrogrids(ordistributedgenerators)andmaingrid.Controlcentreperformsprocessingtodetermineifamicrogridissynchronizedwiththemaingridandcanbesafelyconnectedtocontributeelectricitytothemaingrid.Normally,microgridscandynamicallyconnectanddisconnectfromthemaingridwhichincreasestheriskforpowersystems(ifconnectedinnon-synchronizedstate).Queen'sUniversityBelfasthasalocaltestbedondistributedgenerationandspecificallyresearchingsolutionstoensuresafety,resilienceandcybersecurity.Sincepowersystemsaredistributedinnature,suchsystemsneedtobestudiedinamorerealisticandgeographicallydistributedexperimentalplatform.AsshowninFigure7,theproposedinterconnectedresearchnetworkcanprovidesuchadistributedexperimentalplatformbycombiningresourcesavailableatotherpartnersaswell.ThiswillenableQueen'sUniversityBelfasttoexperimentwithanynewlydevelopedsafetyandsecuritytechnologiesinamorerealisticdistributedpowersystem.Further,partnersinterestedtoconductresearchinthisareabutlackingresourcescanalsobenefitbyaccessingsharedresourcesfromotherpartners.
Figure7:TheDistributedPowerscenario
6.FutureDirections/ConclusionsThiswhitepaperenvisagesaninter-linkednetworkofopentestbedfacilitiesthatwillsupportthegrowingRITICScommunityto:
• betterunderstandtheinterdependenciesbetweendifferentsectors• betterunderstandthesimilaritiesanddifferencesbetweenInformationTechnology(IT)and
OperationalTechnology(OT)• testandpreparefortargetedanduntargetedattacks• providetrainingtoclosetheskillsgap• validatevarioustheoriesabouthowtodealwithnewandunknownthreats• extendunderstandingofsystem-userrelationshipsacrossanarrayofsectors
TheproposalfromRITICSisambitiousandrequiresconsiderableinvestmenttorealisebutrecentlyannouncedNCSCfundingwillallowthedevelopmentofaprototypeinBelfast.WefeelthatcreatingsuchanationalfacilitywillallowtheUKresearchcommunitytomeetthecriteriaoutlinedaboveandrepeatedbelow:
• Fidelity:tobeasaccuratearepresentationoftherealsystemaspossible• Repeatability:repeatedrunsshouldgiveconsistentresults• Measurementaccuracy:observingrunsshouldnotperturbtheoutcome• Safeexecutionoftests:theeffectofatestshouldbecontainedwithinthetestbed
andplaceusinaleadinginternationalpositionforthiswork.
References[1]B.Green,A.Le,R.Antrobus,U.Roedig,D.HutchisonandA.Rashid:Pains,GainsandPLCs:TenLessonsfromBuildinganIndustrialControlSystemsTestbedforSecurityResearch.CSET@USENIXSecuritySymposium2017.
Layer 3 VPN TunnelLayer 2 Light Path
janet
Control Center
Academic Partner
Strathclyde University
Queen’s University BelfastManchester University
[2]R.Candell,D.M.AnandandK.Stouffer:ACybersecurityTestbedforIndustrialControlSystems.ISAProcessControlandSafetySymposium,2014.
[3]H.Holm,M.Karresand,A.VidstromandE.Westring:ASurveyofIndustrialControlSystemTestbeds.NordSec2015,LectureNotesinComputerScience,9417,SpringerVerlag,2015.
[4]C.Siaterlis,A.GarciaandB.Genge:Ontheuseofemulabtestbedsforscientificallyrigorousexperiments.IEEECommunicationsSurveys&Tutorials15(2),2013.
[5]TheHagueSecurityDelta:SecuringCriticalInfrastructuresintheNetherlands:TowardsaNationalTestbed.https://www.thehaguesecuritydelta.com/images/HSD_rapport_Testbed_EN.pdf
[6]D.Balenson,L.TinnelandT.Benzel:CybersecurityExperimentationoftheFuture(CEF):CatalyzingaNewGenerationofExperimentalCybersecurityResearch.http://cyberexperimentation.org/files/2114/5027/2222/CEF_Final_Report_Bound_20150922.pdf
