OpenTestbedsforCNI

AuthorsChrisHankin(editor),ImperialCollegeLondon

DeephChana,ImperialCollegeLondonBenGreen,LancasterUniversityRafiullahKhan,Queen’sUniversityBelfastPeterM3,NationalCyberSecurityCentrePeterPopov,CityUniversityofLondonAwaisRashid,LancasterUniversityandUniversityofBristolSakirSezer,Queen’sUniversityBelfast

1Introduction:Rationale/JustificationIndustrialControlSystems(ICS),composedofcombinationsofhardware,softwareandICTnetworks,orchestratethemyriadoffunctionsneededtoexecutecomplextaskssuchasthedeliveryofutilityservicesandtheoperationofintricateanddisparatemanufacturingprocesses.ICSareexamplesofcyber-physicalsystems–digitalsystemsthataffectandareaffectedby,physicalprocesses–whoseuseisgrowingthroughdevelopmentsinsmart-citytechnologiesandtherapidemergenceoftheInternetofThings.Suchsystemsareincreasinginimportanceastechno-socialcomponentsoftheCriticalNationalInfrastructure(CNI)ofthefutureandastheyextendtheirscope,becomingubiquitous,accessibleandtransformativetowidersocietyandtheeconomy,theneedtounderstandtheirsecuritycharacteristicsalsoincreases.TodatetheResearchInstituteinTrustworthyIndustrialControlSystems(RITICS)activityhasfocussedonidentifyingexistingtechnicalandpracticalproblemsthatsurroundthedevelopmentofsecureandtrustworthyICS.InordertodeveloprealisablesolutionstotheseproblemsRITICShasconductedaresearchprogrammethatincludesworkin:

• Theoryandanalysis• Simulationandexperimentation• Testingandimplementation

Toeffectivelyexecutethismission,theneedforasimulation/labspacewherecomponentsandinstancesofinvestigateddigitalsystemsmaybephysicallyconfiguredfor'closetoreal-world'fidelityisvital.RITICSpartnershavedevelopedsmall-scaletestbedfacilities.Thiswhitepapersurveysthecurrentrangeoffacilities,summarisesthelessonslearnt,presentstheissueswithlinkingthesefacilitiesandconcludeswithaforwardlook.Ourambitionistointerconnecttheexistingsystemstogetherinordertoachievethescaleofreal-worldsystemsandtousethecapabilitiestoaccelerateandincreaseefficiency/effectivenessoftheUKinvestment.Thiswillenableusto

• betterunderstandtheinterdependenciesbetweendifferentsectors• betterunderstandthesimilaritiesanddifferencesbetweenInformationTechnology(IT)and

OperationalTechnology(OT)• testandpreparefortargetedanduntargetedattacks• providetrainingtoclosetheskillsgap• validatevarioustheoriesabouthowtodealwithnewandunknownthreats• extendunderstandingofsystem-userrelationshipsacrossanarrayofsectors

TherewillalsobetheneedtodevelopabusinessmodelforhowtheOpenTestbedsmightoperate.

2.TheUKLandscape

TheLancasterICSTestbedThemostextensivetestbedfacilitiesdevelopedwithinRITICShavebeendevelopedbytheMUMBAprojectattheUniversityofLancaster.Inadditiontotheirlab-basedtestbedwhichcanbeconfiguredinanumberofways,theyhavealsodevelopedatable-topwatertreatmentdemonstrator.

Ahigh-levelviewofLancaster’sICStestbedisshowninFig.1[1].ThearchitectureisbasedonthePurdueReferenceArchitecture.CurrentlysplitacrosssixManufacturingZones,anICSDemilitarisedZone,andanEnterpriseZone(withitsownseparateDemilitarisedZone),allequipmentinthetestbedisphysical(unlessotherwisenotedasVirtualisationPlatforminFig.1).ItisimportanttonotethatLancaster’stestbedhasfocusedonthedevelopmentofsystemsanddevicesacrossLevels0,1,2,3,DMZand4ofthePurduemodel.

Figure1:NetworkDiagramofSecurityLancaster’sICSTestbed

ITRC’sDAFNIprojectTheInfrastructureTransitionsResearchConsortium(ITRC)isaconsortiumof7universities(Cambridge,Cardiff,Leeds,Newcastle,Oxford,SouthamptonandSussex),investigatingwaystoimprovetheperformanceofinfrastructuresystemsintheUKandaroundtheworld.Theirresearchishelpingbusinessesandpolicymakerstoexploretheriskofinfrastructurefailureandthelong-termbenefitsofinvestmentsandpoliciestoimproveinfrastructuresystems.

TheDataandAnalyticsforNationalInfrastructure(DAFNI)projectwillcreateanationalinfrastructuredatabaseforvisualisationandanalysis.Itwillbeashared,securesystemforacademicresearchandaresourceforbusinesses,innovatorsandpolicy-makers.

DAFNIwelcomesideasabouthowitwillworkwithdifferentpartners,andonissuessuchas:security,access,suitablebusinessmodels;andnextsteps.

AkeyfeaturewillbeDAFNI’ssimulationandvisualisationfacilitiestoallowuseofmodelsinamoreflexibleway,enablingthesystemsofsystemsanalysisandincorporatingobservedandsimulateddatasets.DAFNIwillbenefitfromtheexperienceoftheITRC)whichhasbeendevelopingaone-stopdatabaseforUKinfrastructure(NationalInfrastructureSystemsMODel–NISMOD).It’smuchmorethanacurationofdata,andallowsrepresentationofinterdependenciestoinformplanningdecisions,includingviaavisualisationdashboard.AlthoughNISMODcontainsover400datalayers(representingmultiplesectors,demographics,economics),theinfrastructuresectorneedsgreaterdetail,torepresentindividualbuildingsandtodevelopplausibleconnectivitynetworks,whichDAFNIcandeliver.ITRC-Mistralisdevelopingameta-databasetogiveuserstheexperienceofasingleinterface,althoughitbringstogethermanydatabases,andthisisthemodelthatwillbeappliedtoDAFNI.

OnechallengewillbehowtomakeDAFNIsuccessfuloperationally.DAFNI’svisionistobuildanenvironmentwherepeoplecantrydifferentsolutions,whichmeansbeingresponsivetoallusers.ExistingmodelsmightincludeJASMINthatusesthedesktopasaservicetoolusingastandardtoolkit,withnorestrictionsonusers.

5GTestbedsTheDepartmentofDigital,Culture,MediaandSport(DCMS)areinvestingina5GtechnologytestnetworkaimingtoputBritainattheforefrontofthenextwaveofmobiletechnology.

5GresearchinstitutionsatKing’sCollegeLondonandtheUniversitiesofSurreyandBristol,havebeenawarded£16mtodevelopthecutting-edge5Gtestnetworkwhichwillbringacademiaandcommercialcompaniestogethertotrialthetechnologyandmakesurepeopleandbusinessescanrealisethebenefitssooner.

Thistestnetworkwilltrialanddemonstratethenextgenerationofmobiletechnologyandisthefirstpartofafour-yearprogrammeofinvestmentandcollaborationintheGovernment’snew5GTestbedsandTrialsprogramme.

Theuniversitieswillworktogethertocreatethreesmall-scalemobilenetworkswhichtogetherwillformthetestnetwork.Eachnetworkwillhaveanumberoftheelementsexpectedinacommercial5Gnetwork-includingmobilesignalreceiversandtransmittersandthetechnologytohandle5Gsignals-tosupporttrialsofitsmanypotentialuses.

Otheracademicinstitutions,industryandlocalauthoritieswillalsobeabletobidforfurtherfundingtobepartofthisprogrammefrom2018/19onwards.Furtherdetailsonopportunitiesandthefundingavailablearepublishedintheprospectus.

UKCRICTheUKCollaboratoriumforResearchinInfrastructure&Cities(UKCRIC)willprovideleadershipandsupportforthedevelopmentandgrowthofacoordinatedandcoherent,worldclass,UK-basednationalinfrastructureresearchcommunity,spanningatleast14universities.Itwillengagegovernment,cityandcommercialpolicymakers,investors,citizensandacademiainajointventurethatdrivesinnovationandvaluecreationintheexploitationofservicesprovidedbynationalinfrastructure.Throughcentralcoordination,providingafocalpointforknowledgetransfer,UKCRICwillsupportastep-changeinthenation’sapproachtoinfrastructureinvestment.Itwillalsodevelopacommercialresourcethathasconsiderableexportpotentialforaninternationalmarketthatisvaluedat$57trillionintheperiodupto2030.

UKCRICwillunderstandhowtomakethesystemofsystemsthatconstitutesthenation’sinfrastructuremoreresilienttoextremeeventsandmoreadaptabletochangingcircumstancesandcontexts,andhowitcanprovideservicesthataremoreaffordable,accessibleandusabletothewholepopulation.

PETRASThePETRASHubhasfundingforthecreationofanumberofdemonstrators.Theprojectisstilldebatingwhatformtheseshouldtake.

UniversityofBristolWiththeMumbaprojectteam’smovetotheUniversityofBristol,anewICStestbedisbeingsetupthatwillincludemultiplefieldsitesandindustrialprocessestosupportresearchonsecurityofindustrialcontrolsystems,includingbothlegacyandnon-legacydevicesandIndustrialInternetofThings(IIoT).

3.InternationalFacilitiesHolmetal[3]presentanoverviewofinternationalfacilitiesasattheendof2015.Theyidentify30testbedsthatwereeitherplannedorinoperation,almosthalfofwhichwereintheUS.TheyciteSiaterlisetal[4]whopresentthefollowingcriteriathatcybersecuritytestbedsshouldfulfil:

• Fidelity:tobeasaccuratearepresentationoftherealsystemaspossible• Repeatability:repeatedrunsshouldgiveconsistentresults• Measurementaccuracy:observingrunsshouldnotperturbtheoutcome• Safeexecutionoftests:theeffectofatestshouldbecontainedwithinthetestbed

Thesearereasonablerequirementstoexpectofanytestbedfacility.

The iTrust Water testbeds (Singapore) are small-scale networks within a controlled laboratoryenvironment,composedofasmall-scalewaterdistributionnetwork (WADI)anda treatmentplant(SWaT). The testbeds are used for security analysis for water distribution networks, to assessdetectionmechanismsforcyberandphysicalattacks,aswellastounderstandcascadingeffectstootherconnectedsystems.The[iTrust]InternetofThingsAutomaticSecurityTestbed(Singapore)isasmall-scale laboratory composed of GPS simulator, Wi-Fi localization simulator, time simulator,movementsensor,tosimulatethedifferentenvironmentalconditionsinwhichIoTdevicesoperate.Thetestbedsupportsstandardandcontext-basedsecuritytestingandanalysisforIoTdevicesunderrealconditionsagainstasetofsecurityrequirements.Power-Cyber(USA)isasmartgridtestbedwiththepurposetoperformvulnerabilityassessment(i.einspect weaknesses within the infrastructure), design mitigation methods, and develop cyber-physical metrics (i.e metrics combining cyber-physical properties), cyber forensics tools (exploreways to detect cyber-attacks specific to industry protocols and field devices), and securemodels(explorationofinnovativesecurityapproaches).TheUniversity of Illinois atUrbanaChampaign has developed the Cyber-Physical ExperimentationEnvironmentforRemoteAccessDistributedICS(CEER).Asummaryofthiseffort(includedverbatimhere) has been extracted from: https://iti.illinois.edu/research/energy-systems/cyber-physical-experimentation-environment-radics-ceer

“Thegoalofthisprojectistoprovideatestbedonwhichprospectivetechniquesandtoolscanbedeveloped,refined,andvalidatedinacontextwithunprecedentedsystemfidelity.Weareclosingthegapbetweenneedsandstateoftheartthroughatestbed,CEER,thatisinnovativeinseveralways.

CEERbringstotheICSdomainforthefirsttimeproductionqualitysoftwaretoflexibly(andremotely)defineexperiments,configuretestbedresources,andrunexperiments.Itbringsthefruitsofstate-of-the-artmodelingofgridsystemstoprovidesyntheticbutrealisticdynamicgridstate.Itbringscutting-edgeappliedresearchintemporalcoordinationofrealdevices,deviceemulation,andsimulatorsofdiversekindstoenablecreationofexperimentaltopologiesthataremuchlargerthantheensembleofphysicalICSdevicesinthetestbed.CEERbringsbest-of-breedICSsysteminstrumentationandmonitoringtechnologytoenableuserstocloselytracktheresultsoftesting.Itwillbeabletoaccuratelyrepresentthesmartgridinteractionsfromgeneration,transmission,anddistribution.Itwillalsosupporthigh-fidelityexplorationofassetsineachofthesedomains,including,butnotlimitedto,generationassets,gridcomponentsintransmissionanddistributionsubstations,controlcenteroperation,andadvancedmeteringinfrastructure.”

Anapproachtakenbythecolleaguesbehindthistestbedistousehigh-fidelitysimulatorsofthe“physicalworld”,whichallowsforclose-to-trueimpactofcyber-attackstobeaccountedfor.

TheUSNationalInstituteofStandardsandTechnology(NIST)isdevelopingacybersecuritytestbed(seeFig.2[2]).

Figure2:TheNISTTestbed

Theaimistomeasuretheeffectofprevailingstandardsandguidanceontheperformanceofcontrolsystems.Thetestbedisdesignedasaseriesofenclavesthataddressdifferentindustrialsectors.ThetestbedusessimulationwhereappropriatewithHardware-in-the-Loop(HIL)componentssimulatingtheinterfacesbetweensensors/actuatorsandthecontroller.Thedifferentenclavesallowstudyofcontinuousprocesses(suchaschemicalmanufacture),discreteprocesses(suchasautomotiveassembly)andhybridprocesses(suchaspharmaceuticalmanufacture).Performanceismeasuredusingappropriatetechnicalperformanceindicatorsfortheprocesses.

TheDepartmentforBusiness,EnergyandIndustrialStrategyCivilNuclearTeamisintheplanningphaseregardinganupcomingjointexercisewithEstonianMODofficials.TheEstonianofferisanetworkdefenceexercisescenariomakinguseofafully-equippedcybertestrangeinTallinn,andwillinvolveparticipantsfromacrossthecivilnuclearsector.TheNationalCyberSecurityCentrewillalsohaveinvolvementintheexerciseusingtheEstonianDefenceForcesCyberRange,consistingofasimulatedofficenetworkconsistingoftypicalserversandworkstationsaswellasfacilitiessupportsystemswillbeimplemented.

HitachiaredevelopingaSecurityTrainingArena(SeTA)attheirOmikaWorksinJapan.Theemphasisofthiscentreistotrainoperatorshowtodealwithcyberincidentsinanuclearpowerplant.TheyplantorunjointexerciseswiththeUK(andpossiblyUS)in2018.TheyhavehadpreliminarydiscussionswithImperialCollegeLondonandRoyalHollowayUniversityofLondonaspotentialacademicpartnersinthisprogramme.

ThelargestsecurityclusterinEuropeissituatedinTheHagueSecurityDelta.In2015,theypublishedaproposalforanational,multi-sectortestbed[5].Atthetimeofwriting,theyarestillrecruitingpartnerstoassistintheconstructionofthefacility;theinitiativeissupportedbyTNO,KPNNVandtheMunicipalityofTheHague.TheminimumrequirementsfortheDutchnationaltestbedareasfollows:

• Theplatformshouldhosttestlabsformultiple,differentcriticalinfrastructuresectors• Theplatformshouldgenerateknowledgethatcanbeusedtocreatesolutionsforcritical

infrastructureequipment• Itshouldbeavailablefortrainingofinformationsecuritystaffonthreatsandexploits• Thetestbedshouldfacilitatethecreationofanetworkofhighlyqualifiedinformation

securitystaff• Thetestbedfacilityshouldperiodicallyproduceconfidentialreportsaboutnewlydiscovered

threatsandvulnerabilities• Thetestbedfacilityshouldprovideopenandfreelyavailablesecurityreportswiththe

securitysolution• Thetestbedfacilityshouldturnsecurityrequirementsintonewindustrystandards• Thetestbedfacilityshouldeducatecriticalinfrastructurecompaniesinbestpracticesand

lessonslearnedfromacrossallsectors• Thetestbedfacilityshouldestablishcooperationandinformationsharingamong

participatingpartners

TheserequirementsoverlapsignificantlywiththeambitionthatweoutlinedinSection1above.

4.DesignIssuesandLessonsLearntThekeydesignissuesandlessonslearntfromtheconstructionoftheLancastertestbed[1],whichalsofindechoesintheothercitedpapersare:

1. Theneedtoinclude,eitherphysicallyorvirtually,adiverserangeofdifferentdevices(vendorsandversions)

2. Theneedforscaletoprovidefaithfulrepresentationsofrealsystems3. Appropriatemechanismstomanagethecomplexityoftheinfrastructure

DiversityAneffectivetestbedshouldbeabletomimicavarietyofICSsetups.Keyquestionsinclude:

1. Selectionofdevicesandprotocolsforinclusion;2. Providingdifferentconfigurationsofdevices/manufacturerstypicalinICSsettings;and3. Balancingdeviceandprotocoldiversityagainstotherrequirements,suchasthe

implementationofthephysicalprocessitself.

ExperienceswithintheLancastertestbedhavehighlightedthat[1]:

• Deviceandtechnologyselectionsshouldbemarket-driven;• Fieldsitesinatestbedshouldrepresentdifferentreal-worldscenariossuchashomogeneity

andheterogeneityofvendorsaswellascombinationsoflegacyandnon-legacydevices;• Processdiversitycanhelpmodelstealthattacksthatexploitphysicalaspectsoftheprocess

butthatsuchprocessdiversitymaybetraded-offinfavourofdiversityofdevicesandfieldsites.

ScaleSoftwaredoesnotprovidesimulationsofmanyessentialtypesofdevices,i.e.fromdifferentvendorsorthesamevendorbutdistinctiveversions.Theaccuracyandreliabilityofsuchsimulationsinmimickingreal-lifeoperationsalsoremainanissue.Therefore,whilethecostofphysicalequipmentcanbealimitingfactor,thebenefitsitcanbringinrelationtoexperimentalrigourisanoverridingconstraint.Ontheotherhand,virtualisationandVLANscanprovideeaseofintegrationandscalingofthetestbedinfrastructure[1].

ComplexityAlthoughtheunderlyingarchitecturemaybecomplexandinvolveanumberofnetworkzones,thisshouldbeastransparenttotheuseraspossible.Transparencycanbeachievedbyprovidingasinglepointthroughwhichaccesstoandextractionofdatafromthedifferentzonescanbemanaged.AsecondlessonlearntbytheLancasterteam[1]isthenecessitytocreateandmaintaingooddocumentationofthetestbedasitevolves.

FurtherLessonsTheNSFreportonCybersecurityExperimentationoftheFuture[6]providesadetailedroadmapforthedevelopmentoffutureexperimentationinfrastructureoverthenear-term(3years),mid-term(5years)andlong-term(10years).Thereportalsoreviewstheexperienceof46US-basedexperimentalfacilities.

Thetop5recommendationsfromthereportareasfollows:

• Focussingonmultidisciplinaryexperimentationdrawingonboth“hard”sciencesandsocialsciencewillhavethegreatestimpactinacceleratingcybersecurityexperimentationinthenearterm.

• Theabilitytoaccuratelyrepresentfullyreactionarycomplexhumanandgroupactivityinexperimentswillbeinstrumentalinenvironmentsthatrealisticallyrepresentreal-worldsystems.

• Creatingopenstandardsandinterfacesisamid-termpriority.• Researchanddevelopmentusingthelatestadvancesindatascienceisneededtocreate

reusable,extensible,validatedexperimentdesigns.• Researchinfrastructuremustbeusablebyabroadrangeofresearchersandexperts,notjust

restrictedtocomputerscienceresearchers.

5.LinkingTestbedsMostUKacademicinstitutionsandresearchcentresareprovidedconnectivitybyJanet,ahigh-speed,secureandreliableworld-classnetwork.Janetprovidesatleasta10Gb/sphysicallinkandaClass-BIPaddresspool,enablingalltypesofInternetserviceswithinUKacademiccampuses,includinglow-latencyVoiceoverIP(VoIP).However,experimentallabfacilitiesandresearchtestnetworksaresignificantlyconstrainedintakingadvantageoftheJanetinfrastructure.InordertoavoidsecurityandQualityofService(QoS)relatedthreatstotheJanetnetwork,researchlabfacilitiesare,inmostcases,disconnectedandrelyonexternalmulti-megabitADSLlinesvialocalISPproviders.Theselimitationsnotonlyconstrainresearchcapabilitieswithintheseinstitutions,butalsoimpairnationalandinternationalcollaborationsthatrequirehigh-speedconnectivityamongstcollaboratingpartners.

Anotherkeyfactorthatlimitstheresearchcapabilities,qualityandeffectivenessisthelimitedavailabilityofresourceswithinacademicinstitutionsandresearchcentres.SettingupanexperimentallabfacilityorresearchtestnetworkisextremelyexpensiveespeciallyintheICSandSCADAdomainduetotheneedforexpensivedevices/equipment.Thenon-availabilityofstate-of-the-artexperimentalresourcessignificantlylimitsresearchpotentialofindividualacademicinstitutions.

Theaimforlinkingtestbedsistoenableallpartnerinstitutionswithleadingedgeresearchcapabilities,experimentallabfacilitiesandtestnetworksbysharingresourcesoversecureandreliablehigh-speedJanetinfrastructure.TheteamatQUBhaverecentlyproposedanapproachwhichisuniqueofitskindbyinter-linkinglabfacilitiesofalluniversitiesacrosstheUKasshowninFigure3.ThisproposaldoesnotjustfocusonlinkingICSandSCADAfacilitiesbutproposesamoregeneralnetworkoftestbeds.

AshighlightedontherightinFigure3,thesharingnaturewillenableallparticipatinginstitutionstobenefitfromthesamesearchfacilitiesandhaveaccesstotest/experimentalnetworkswhichtheywerelackingindividually.TheproposedresearchnetworkwillbebuiltuponJanet’snetworkinfrastructureusingconfigurablemulti-gigabitVPNtunnels,providingconnectivityofupto10Gb/samongstthepartners,whilefacilitatingstrictisolationfromeachnode’smaincampusnetwork;asimilararchitectureisalreadyunderevaluationtoallowexternalconnectivitytotheLancastertestbed.Centralisednetworkadministrationandmanagementwillprovideprojectspecificconfigurationofthenetwork(topology,bandwidth)andexternalconnectivitytonationalandinternationalpartners,andtheInternet,viaasecuregatewayusingJanetandthird-partyISPs.The

baselinearchitecturewillbelaidoutassuchthatthenetworkcanbescaledtoexpandbeyondthecurrentpartners,capableofservicingtheUKacademicresearchcommunityformanyyearsahead.

Figure3:AProposedUKNetwork

DirectBenefitstoUKUniversitiesandResearchCentresTheacademicandindustrialpartnerswillbeabletotakeimmediateandfulladvantageoftheproposedconnectivitybyprovidingthem:(i)anextendedresearchnetworkinfrastructureforexperimentalstudies,(b)accesstothecollectivepoolofstate-of-the-artexpensivetechnologies,tools,testbedsanddatasets,whicharecurrentlyonlyavailabletotheowninginstitution,(c)providethenecessaryinfrastructureforgeneratingresearchspecifictrafficsamples,logfilesandotherspecialistdatasets,andthefacilitytosharealargerepertoireofexistingdatasetsamongstthepartners,(d)limitedaccesstospecialistnetworkingskillsanddomainknowledgebytakingadvantageofthenetworkanditsdedicatedstaff.

Thescaleanddiversityoftheproposedresearchnetworkandtestfacilitiesprovidesuniqueresearchopportunitiesforendusers,suchasenterprisesandcorporationsrelyingonlargeITnetworksandITsecurity.Theproposedresearchnetworkwillprovidenumerousbenefitsforpartnersbysharingorprovidingaccesstoexpensiveandrareresources,accesstomorerealisticexperimentalenvironmentandimprovingresearchcollaboration.

QUBhasanextensiveexperimentalnetworkandtestlabinfrastructurecloselycoupledtoasystempenetrationtestandtraininglab.Theinter-linkedresearchnetworkwillextendthattestcapabilityandprovideamorerealisticanddistributedICSsystemtoexperimentwith.Inadditiontothe

Aberdeen

Dundee

EdinburghGlasgow

Belfast

Liverpool Manchester

Leeds

Sheffield

LeicesterAberystwyth

Birmingham

Cambridge

Cardiff London

Southampton

Plymouth

Bristol

Oxford

Canterbury

Lancaster

Layer3VPNTunnelLayer2LightPath

janetQUB

UoE

USW

UE

DMU

UB

Internet

ISPJanet

Gateway

Network AdministrationManagement and SecurityTraffic monitoring and acquisition

RHUL

Lanc.Bris.

benefitsofscalethatwillaccruetoexistingRITICStestbeds,suchastheoneatLancaster,thenetworkwillalsobenefitothernewRITICSpartners.Forexample,theplannedtestbedatBristolwillbelinkedintothenetworkaswellasthoseuniversityfacilitiescurrentlysupportedbyAirbus.ThecollaborativeprojectbetweentheUniversityofSouthWales(USW)andAirbusDefenceandSpacecalledSCADACyberSecurityLifecycle(SCADA-CLS),istargetingthedevelopmentofacyberforensiccapabilityforSCADAprocesscontrolsystems.Theinter-linkedresearchnetworkwillprovideanextendedICSnetworkthatUSWcaneffectivelyutiliseforforensic/incidentmanagementtriageprocessmodelling,andthedevelopmentofSCADAforensictoolsfordataacquisition,incidentmanagementandsituationalawareness,usingSCADAtestfacilitiesatQUBanditsFP7/H2020partners.DeMontfortUniversity’s(DMU)CYRANcyberrangetechnology,whichwillbedirectlyaccessiblebyallpartners,providesaplatformforcyber-attack/defencescenariosforexperimentalresearchandforeducationalgamesthatincludephysicalartefactssuchasPLCcontrolledproductionlinesandfiltrationsystems.DMUwillbeabletoextendtheCYRANcapability,accessingPLCcontrollersatQUB,USWandEuropeanpartners.SCADAandothertypesofICSrelatedlargedatasetscannowbegenerated,takingadvantageoftheadditionalphysicalresourcesfromthepartners’testbeds.ThisdataisdirectlyrelevanttoexistingprojectswithinDMUonSCADAForensics,undertakenwithAirbusGroupInnovationsaswellasresearchonPrivacyMetricsandIncidentResponsemanagement.

Bycombiningmanytestnetworkswithuniqueproperties,moregeneralcybersecurityresearchprojectswillalsobenefit.Logfilesfromnextgenerationfirewalls(ngFW)withintheproposedresearchnetworkwillbeusedtoanalysemalicioustrafficinLANnetworks;workingwithmultiplengFWdatawillenabletheanalysisacrossawideareanetwork.Akeybenefitisthegenerationoflargelogfileswithintheexperimentalnetworkwithoutbeingconstrainedbytheprivacyandethicalchallengesoflivecampusnetworks.Dedicatedmonitoringandinterceptiontechnologywithintheproposedresearchnetworkwillprovideadvancedtrafficvisibilityandpacketprocessingcapabilityformanyprojects.Theproposedresearchnetworkwillallowpartners(a)tofurtheranalyserepetitiveexternalattackstotheirITinfrastructurebyreplayingattackpatterns,(b)usecross-sitetestcapabilitiestoundertakestressandpenetrationtestingonneworexperimentalsecurityandnetworkappliances,and(c)assessnewcybersecurityarchitecturesandthreatmitigationstrategiesoncorporatenetworksandwebsites.

Datasetandtesttrafficgenerationandsharingisoneofthemostimportantandchallengingtopicsinnetworkandcybersecurity.Availabledatasetssuchasinterceptedtrafficareconstrainedandinmostcasesrelevanttoaspecifictypeofthreat.Privacyandethicalconsiderationspreventtheuseofanyintercepteddata,suchasfromaUniversitycampusnetwork.Furtherconstraintsarethatmalware,APTandDDoSrelatedprojectsrequiresfreshdatasetsandtrafficcontainingtargetedthreatsinordertounderstandtrafficpatternsrelatedtothreats,andforoptimisingdetectionalgorithmssuchasmachinelearningclassifiers.Theproposedresearchnetworkbringstogetherhighlydiversetestnetworksatascaleandthetrafficcapacityofalargenetwork,providingauniqueopportunityforgeneratingtailoreddatasetsandsampletraffic.

Transportexperimentallabfacilitiesarequiteexpensivetoestablishandonlyfewuniversitieshaveadvancedtestbeds.Theinterconnectedresearchnetworkwillbeofsignificanceforimprovingcollaborationamongstacademicinstitutionsandeffectivelysharingtheirtransportlabfacilities.BirminghamCentreforRailwayResearchandEducation(BCRRE)oftheUniversityofBirmingham(UoB)hassignificantexperimentallabfacilitiesforresearchinaddressinggrandsystem-wideaswellascomponentlevelchallenges.UoBrailwayresearchcoversvariousaspectsincludingsafety,operationsandmanagement,dataintegrationandcybersecurity.Recently,theUKRailResearch

andInnovationNetwork(UKRRIN)researchcentrehasbeenestablishedforsupportingnewinnovationsinrailtransport.UKRRINaimistobringtogetherexistingfacilitiesatdifferentacademicinstitutionsandaccelerateinnovationandnewproductdevelopmentintherailindustry.AspartofUKRRIN,UoBwillcarryoutresearchindigitalrailsystemscoveringcybersecurity,trafficmanagementandrailwayconditionmonitoringandsensing.UniversityofNewcastle(UoN),LoughboroughUniversity(LU)andUniversityofHuddersfield(UoH)withinUKRRINwillcollaborateonhighvaluerollingstocksystems,assetoptimisationandthrough-lifemanagementandenergymanagement.Whereas,UniversityofSouthampton(UoSA),UniversityofSheffield(UoS),UniversityofNottingham(UN)andHeriot-WattUniversity(HWU)arecarryingoutresearchonrailwayinfrastructurewithinUKRRIN.TheproposedinterconnectedresearchnetworkwillbethemediumenablingallpartnersofUKRRINtocollaborateeffectivelyandshareexperimentalresources.

ProposedSystemArchitectureTheproposedinter-linkedresearchnetworkwouldbedevelopedinmultiplephases,takingadvantageoftheavailableJanetconnectivityandsparebandwidthcapacityoftheacademicinstitutes.

Figure4outlinestheoverallnetworkandtestbedarchitectureamongstthepartners.Aninitialphasewouldtargetthedevelopmentofthebasicoverlayarchitectureontopoftheexistinglayer3Janetconnectionviamulti-gigabitVPNtunnelsandtheestablishmentofthenetworkwiththenecessarynetworkadministrationandmanagementtoolsandsupportresources.Acontrolcentrewithnetworkadministrationandmanagementtoolswouldbeestablished,responsiblefortheadministrationandmanagementofthelinksamongstthepartners,JanetandtheexternalconnectivitytotheInternet.Phase-2developmentwouldprovideadditionalphysicallinkcapacitiesandexternalconnectivitytointernationalandindustrialpartners.

Figure4:TheProposedArchitecture

Janetlayer2LightPathprovidesthelowestcostandhighest-bandwidthconnectivity,fortheproposednetworkitalsointroducessignificantchallengesprovidingthenecessarylayer3networkinfrastructureandsupportingcybersecurity(malware,DDoS,cloud),IoT,andIndustrialcontrol

relatedtestlabs.NumerousSDNandcloudnetworktestbedscantakefulladvantageofthelayer2bandwidthsuchasstreamingterabytesofdatabetweenbig-datalabs.

RequirementsforSecureInter-linkingofDiverseTestbedsTheobjectiveistodevelopaplatform,basedonasecureoverlaynetworkarchitecture,forinterconnectingvariousacademicandindustrialtestbedsintoalargerUKwideresearchnetwork.AsdepictedinFigure4,suchaprivateoverlaynetworkapproachhasthreebasicrequirements:(i)networkconnectivitykit,(ii)centralizedoperationsandmanagementand(iii)highspeedJanetnetwork.

NetworkConnectivityKitThenetworkconnectivitykitenablesremotetestbedsitesfrompartnerstobeconnectedtotheprivateoverlaynetworkasshowninFigure4.Thesolutionisscalableandnewtestbedscanbeeasilyintegratedwithinthenetworkwithoutmajortechnicalsupport.Tobecomepartoftheinterconnectedphysicaltestbedinfrastructure,eachacademicorindustrialpartnershouldbeprovidedwithanetworkconnectivitykitorrack-mountedkit.Thebasicarchitectureoftherack-mountedkitisshowninFigure5andconsistsof:

• AFirewall/Router/VPNwhichwillbemanagedfromtheCSITTestNetworkManagementcentre.

• Adistributionswitchthathasportmirroringcapabilitiestopermittrafficcapture.• Trafficdatastoragecapability.Terabytesofnetworktrafficdatamayneedtobecaptured

andstoredforlateranalytics.

Figure5:ArchitectureoftheNetworkConnectivityKit

ThenetworkconnectivitykithasaVPNclientandappropriatelydimensionedcommunicationandstoragehardware.ItistailoredfortheproposedresearchnetworkcomprisedofappliancesforatraditionalIPnetworkandanSDNnetwork,capableofsupportingadvancedfirewall,andVPNtunnelwithVLANsegregationcapability.

OperationsandManagementTheproposedresearchnetworkhascentralizedmanagementformanagingconnectivitybetweendistributedtestbedsites,networkaccesscontrolanddataacquisition.Italsohasadata-set(sampleddataandtrafficpatterns)repositorywithpostprocessing,indexingandaccesscontrol.Aneffective

managementandadministrationstructureisessentialtoensurethesuccessoftheproposedresearchnetworkanditsefficientutilisationbythepartnersandthewiderresearchcommunity.

ThecentralizedoperationsandmanagementsystemwillbehostedattheCentreforSecureInformationTechnologies(CSIT)datacentre.TheproposedresearchnetworkwillbemanagedbyQueen’sUniversityBelfast,aspartofCSIToperationandmanagementinfrastructure,incollaborationwithallpartners.

HighspeednetworkThehighspeedlinkcapacitieswillbeleasedfromJanetwhichwillactasthebackboneandfabricoftheproposedresearchnetwork.ThemajorityofcostinthedevelopmentofproposedresearchnetworkisassociatedwiththeleasingofcommunicationlinksfromJanetandprovidingthenetworkconnectivitykittoeachpartner.GoingbeyondthecurrentQUBproposal,itshouldalsobeconsideredhow4Gtechnologies,asusedatLancaster,canbeincorporatedintothenetworkinasecure,reliable,andmanagedformat.TherecouldbeafurtherextensiontowardsPSTN/GSMservices,inwhichlegacydial-uptechnologiesmayalsobeapplied.

UseCaseExamplesInter-linkingexperimentalresourcesfromacademicandindustrialpartnersmakestheproposedresearchnetworkquiteheterogeneousconsistingofdiversetestbedsinalldifferentresearchareas.Basedontheresearchtopic,apartnercanrequestresourcesinaspecificdomainfromcontroloperationsandmanagementcentre.TheControlcentrewillcreateasecuresegregatedVLANwithdedicatedexperimentalresourcesbasedontherequest.Theallocatedresourcescanthenbeexploitedbythepartnerstoexperimentanddetermineeffectivenessoftheirdevelopedtechnologiesandresearchtools.Toillustratetheutilityofthenetwork,thissectionpresentstwoICSusecaseswheretheproposedresearchnetworkcanbeutilized.

DistributedIntrusionDetectionandPreventionAnIntrusionDetectionSystem(IDS)monitorsanetwork/systemformaliciousactivitiesorviolationofpoliciesandraisesalerts.Whereas,IntrusionPreventionSystem(IPS)complementsIDSbyalsotakingdefensiveactionswhenamaliciousactivityisdetected.SeveralacademicinstitutesandresearchcentresareactivityinvolvedinIDS/IPSresearchtoimprovedetectionefficiencyandeffectivelyhandleemergingthreats.NISTpublishedrecommendationsthatIDS/IPSsystemsshouldbehybrid,distributedinnature,havedecentralizeddecisionmakingandcentralizedmanagementandrefinementofdetectedevents.ThehybridIDS/IPSsystemsperformbothhost-basedmonitoringaswellasnetwork-basedmonitoringformaliciousactivitiesdetection.Thedistributednaturesuggestsmultiplesensorstobedeployedinsysteminsteadofrelyingonasinglesensorforredundancyandbettermaliciousactivitiesdetection.

SeveralICSsystemsaredistributedinnaturee.g.,powersystems.ToinvestigateIDS/IPStechnologiesforadistributedICSnetwork,apartnercanrequestresourcesfromthecontrolcentreofproposedinterconnectedresearchnetwork.ThepartnerwillbenefitfromnothavingitsownbututilizingsharedICStestbedsfromotherpartnerinstitutes.Thiswillenablethepartnertocontinueresearchinthistopicevenifitislackingequipment.

Figure6:TheDistributedIDSscenario

RealisticExperimentalPowerSystemsPlatformTheproposedresearchnetworkcanbeusedtoconductresearchinamorerealisticdistributedexperimentalplatform.E.g.,powersystemsarehighlydistributednowadaysduetodevelopmentofrenewableandgreenenergysources(e.g.,windfarms,solarpanels,etc).Thistrendisbecomingmoreandmorecommonandgreenelectricitysourcesarepredominantlylocatedatgeographicallyisolatedareas.Severaluniversitiesareconductingresearchondistributedgenerationandtransmission,microgridsandsubstationsincludingQueen'sUniversityBelfast,ManchesterUniversityandStrathclydeUniversity.Distributedgenerationandintegrationintomaingridtakesbenefitfromsynchrophasortechnology.SynchrophasortechnologyincludesacontrolcentrethatreceivesGPStimestampedelectricalmeasurementsfrommicrogrids(ordistributedgenerators)andmaingrid.Controlcentreperformsprocessingtodetermineifamicrogridissynchronizedwiththemaingridandcanbesafelyconnectedtocontributeelectricitytothemaingrid.Normally,microgridscandynamicallyconnectanddisconnectfromthemaingridwhichincreasestheriskforpowersystems(ifconnectedinnon-synchronizedstate).Queen'sUniversityBelfasthasalocaltestbedondistributedgenerationandspecificallyresearchingsolutionstoensuresafety,resilienceandcybersecurity.Sincepowersystemsaredistributedinnature,suchsystemsneedtobestudiedinamorerealisticandgeographicallydistributedexperimentalplatform.AsshowninFigure7,theproposedinterconnectedresearchnetworkcanprovidesuchadistributedexperimentalplatformbycombiningresourcesavailableatotherpartnersaswell.ThiswillenableQueen'sUniversityBelfasttoexperimentwithanynewlydevelopedsafetyandsecuritytechnologiesinamorerealisticdistributedpowersystem.Further,partnersinterestedtoconductresearchinthisareabutlackingresourcescanalsobenefitbyaccessingsharedresourcesfromotherpartners.

Figure7:TheDistributedPowerscenario

6.FutureDirections/ConclusionsThiswhitepaperenvisagesaninter-linkednetworkofopentestbedfacilitiesthatwillsupportthegrowingRITICScommunityto:

• betterunderstandtheinterdependenciesbetweendifferentsectors• betterunderstandthesimilaritiesanddifferencesbetweenInformationTechnology(IT)and

OperationalTechnology(OT)• testandpreparefortargetedanduntargetedattacks• providetrainingtoclosetheskillsgap• validatevarioustheoriesabouthowtodealwithnewandunknownthreats• extendunderstandingofsystem-userrelationshipsacrossanarrayofsectors

TheproposalfromRITICSisambitiousandrequiresconsiderableinvestmenttorealisebutrecentlyannouncedNCSCfundingwillallowthedevelopmentofaprototypeinBelfast.WefeelthatcreatingsuchanationalfacilitywillallowtheUKresearchcommunitytomeetthecriteriaoutlinedaboveandrepeatedbelow:

• Fidelity:tobeasaccuratearepresentationoftherealsystemaspossible• Repeatability:repeatedrunsshouldgiveconsistentresults• Measurementaccuracy:observingrunsshouldnotperturbtheoutcome• Safeexecutionoftests:theeffectofatestshouldbecontainedwithinthetestbed

andplaceusinaleadinginternationalpositionforthiswork.

References[1]B.Green,A.Le,R.Antrobus,U.Roedig,D.HutchisonandA.Rashid:Pains,GainsandPLCs:TenLessonsfromBuildinganIndustrialControlSystemsTestbedforSecurityResearch.CSET@USENIXSecuritySymposium2017.

Layer 3 VPN TunnelLayer 2 Light Path

janet

Control Center

Academic Partner

Strathclyde University

Queen’s University BelfastManchester University

[2]R.Candell,D.M.AnandandK.Stouffer:ACybersecurityTestbedforIndustrialControlSystems.ISAProcessControlandSafetySymposium,2014.

[3]H.Holm,M.Karresand,A.VidstromandE.Westring:ASurveyofIndustrialControlSystemTestbeds.NordSec2015,LectureNotesinComputerScience,9417,SpringerVerlag,2015.

[4]C.Siaterlis,A.GarciaandB.Genge:Ontheuseofemulabtestbedsforscientificallyrigorousexperiments.IEEECommunicationsSurveys&Tutorials15(2),2013.

[5]TheHagueSecurityDelta:SecuringCriticalInfrastructuresintheNetherlands:TowardsaNationalTestbed.https://www.thehaguesecuritydelta.com/images/HSD_rapport_Testbed_EN.pdf

[6]D.Balenson,L.TinnelandT.Benzel:CybersecurityExperimentationoftheFuture(CEF):CatalyzingaNewGenerationofExperimentalCybersecurityResearch.http://cyberexperimentation.org/files/2114/5027/2222/CEF_Final_Report_Bound_20150922.pdf