WhitePaper_MPLS and NERC CIP Compliance BellLab

T E C H N O L O G Y W H I T E P A P E R

Power utilities supporting bulk electric systems (BES) must comply with the Critical

Infrastructure Protection (CIP) requirements specified by the North American Electric

Reliability Corporation (NERC). Specifically, network endpoints, such as routers and

switches that access communications networks at BES locations, are critical cyber assets

(CCAs) and must be protected within electronic security perimeters (ESPs).

This paper shows that MPLS-based networks provide secure, reliable, efficient, flexible

and cost-effective communication between CCAs at different BES locations, as well as

between CCAs and other smart grid elements. Even if utilities decide to take advantage

of the currently available exemption that does not require systems using non-routable

protocols to be protected within ESPs, MPLS networks can be used to emulate all

necessary non-routable protocols over a single networking infrastructure.

We provide an analysis of the current state of the NIST Smart Grid Cyber Security Strategy

and Requirements and discuss how they can be applied to MPLS endpoints in order to

satisfy the NERC CIP cyber security requirements. We also demonstrate how the ITU-T

X.805 security standard can be used to depict the compliance level of a CCA, as well as

the entire ESP.

Alcatel-Lucent offers a family of MPLS routers with a broad range of security features necessary

to provide the defense-in-depth mandated by the NERC CIP cyber security requirements.

Achieving NERC CIP* Compliancewith Secure MPLS NetworksA Bell Labs Memorandum

Ahmet Akyamac, Ph.D., Jayant Deshpande, Ph.D., Andrew McGee, CISSP, GREM, GCIH

* Critical Infrastructure Protection (CIP) requirements from the North American Reliability Corporation (NERC) standards (SectionsCIP-001 through CIP-009). The NERC standards are available at www.nerc.com/files/Reliability_Standards_Complete_Set_2009Dec3.pdf

Table of contents

1 1. Introduction

2 2. Reference Architecture

2 2.1 Key Definitions

2 2.2 Basic Communication Architecture

3 2.3 Extended Reference Architecture

4 3. Communication over MPLS networks

4 3.1 MPLS Architecture

5 3.2 Converged MPLS Networks

6 3.3 Additional MPLS Features

6 4. Interim NERC CIP Compliance with MPLS-based Non-Routable Protocol

8 5. MPLS is the Right Choice with or without the Exemption

8 6. ESP Security Implementation

8 6.1 Requirements Overview

9 6.2 ESP Identification and Protection

9 6.3 System Security Management

9 6.4 Technical Guidance for Compliance with NERC CIP Requirements

10 7. Using ITU-Ts X.805 Security Standard to Secure the Smart Grid

11 8. Threats to the Electronic Security Perimeter

12 9. Potential Vulnerabilities in the ESP

12 10. Mitigations for ESP Vulnerabilities and NERC CIP Compliance

14 11. Conclusions

15 12. References

16 13. Acronyms

17 Appendix A. MPLS Architecture

19 Appendix B. Additional MPLS Features

21 Appendix C. Technical Guidance for Compliance with NERC CIP Requirements

24 Appendix D. The X.805 Security Dimensions

25 Appendix E. Potential Vulnerabilities in the Power Grid

1Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper

1. Introduction

The Reliability Standards for the Bulk Electric Systems of North America1, specified by the NorthAmerican Electric Reliability Corporation (NERC), includes requirements for Critical InfrastructureProtection (CIP) for compliance by electric power utilities in protecting the critical cyber assets(CCA) of bulk electric systems (BES). All hardware, software, data systems, and network elementsat bulk generation stations, transmission substations, and utility data and control centers mustcomply with the NERC CIP requirements.

This paper shows that MPLS-based networks provide secure, reliable, efficient, flexible and cost-effective communication between the CCAs at different BES locations, as well as between theCCAs and other smart grid network elements.

NERC requirements [1] define the Electronic Security Perimeter (ESP) as a logical bordersurrounding a network to which CCAs are connected and access must be controlled. In most cases,an ESP will include CCAs at a single BES location connected over a LAN. Any system (e.g. arouter) that uses a routable protocol (such as IP) is, by definition, considered a CCA and must beincluded in an ESP. Consequently, a communication system in the BES that does not use a routableprotocol would not be considered a CCA. This loosened requirement is referred to as NERC CIPsnon-routable protocol exemption (Examples of non-routable protocols include PDH/SONET,Ethernet or Frame Relay). Therefore, networking systems providing connectivity with a non-routable protocol can reside outside of an ESP, and are not subject to NERC CIP requirements.We also show that MPLS networks can natively and effectively support communication over manynon-routable protocols; therefore a utility does not need to deploy multiple networks with differentnon-routable protocols.

It is believed by many that NERC CIPs non-routable protocol exemption (called the exemptionthroughout this paper) has been deliberately allowed by NERC to facilitate timely NERC CIPcompliance without substantial immediate investment. Future revisions of the NERC CIP requirementsmay require all communication systems at a BES location to be CCAs, removing the current (implied)exemption of systems with non-routable protocols.

In addition, this paper will show that MPLS networks facilitate secure implementation and NERCCIP compliance with or without the exemption.

The reference architecture relevant to the NERC CIP requirements is presented in Section 2.In Section 3, we describe key features of MPLS infrastructure and emulation of communicationprotocols and services. Section 4 establishes the applicability and advantages of supporting non-routable protocols over MPLS infrastructure, leading to compliance of the current requirementswith the exemption on non-routable protocols. Section 5 details MPLS network essentials thatsupport utility applications and NERC CIP compliance, even when the exemption is removed fromthe specifications.

The remainder of this paper discusses the impact of removing the non-routable protocol exemptionwould have on compliance requirements. An overview of the nine NERC CIP requirements isprovided in Section 6, along with guidance for satisfying the requirements technical aspects.Section 7 describes ITU-T Standard X.805 [2] and how it can be used to measure compliance levelsof a cyber asset or entire ESP. Sections 8 and 9 lists locations, threat types, and potentialvulnerabilities to the bulk electric system. Section 10 describes countermeasures that can mitigatethose vulnerabilities. Finally, a summary and our conclusions are presented in Section 11.

For convenience, several Appendices at the end of the document present additional information onMPLS features, X.805, and other security aspects related to the main body of the document.

1 http://www.nerc.com/files/Reliability_Standards_Complete_Set_2009Dec3.pdf

2. Reference Architecture

Before presenting the reference architecture, a few relevant terms from the NERC CIP standard [1]are introduced.

2.1 Key DefinitionsThe NERC CIP requirements [1] (more correctly the Regional Reliability Organization) define aBulk Electric System (BES) as the electrical generation resources, transmission lines, interconnectionswith neighboring systems, and associated equipment, generally operated at voltages of 100 kV orhigher. Radial transmission facilities serving only load with one transmission source are generallynot included in this definition. Thus, cyber assets at most distribution substations, and thedistribution feeders, are not covered by NERC CIP requirements.

As defined in the standards, Critical Assets are defined as facilities, systems, and equipmentwhich, if destroyed, degraded, or otherwise rendered unavailable, would affect the reliability oroperability of the BES. Critical Cyber Assets (CCAs) are programmable electronic devices andcommunication networks including hardware, software, and data that are essential to reliableoperation of critical assets. Hardware, software, data systems, and networks at utility controlcenters, bulk power stations, and transmission substations are examples of CCAs2. Distributionsystems, AMI systems, and their interconnection networks are not.

The Electronic Security Perimeter (ESP) is a logical border surrounding a network to whichCritical Cyber Assets are connected and access is controlled. As a practical matter, an ESP will beconfined to a physically protected building or space within. Communication links/networksconnecting discrete ESPs are not considered part of the ESPs3, so routers and switches in theseconnecting networks are not CCAs. However, network endpoints on equipment within an ESPfunctioning as access points to the ESP are considered a CCA and must be secured.

2.2 Basic Communication ArchitectureThe network architecture in Figure 1 illustrates concepts applicable to NERC CIP requirements.

Figure 1. Example Reference - Communication Architecture for a Bulk Power System

2 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper

ESP(Electronic Security

Perimeter)

(Utility) Data and Control Center

(Transmission) SubstationBulk Power Station

LAN

D

P

Routable Protocol ( ie, IP)

T

COMMUNICATIONNETWORK

2 See Requirement CIP-002-B.R1.2 in [1]3 See Requirement CIP-005-B.R1.3 in [1]

The host systems (those attached to protection elements, Supervisory Control and Data Acquisition(SCADA) systems, control and monitoring servers, etc.), switches, routers, and other communicationgear at a BES location are essential for reliable operation of the BES, thus constituting the CCAs.These systems, with their interconnecting LAN, are bound by the ESP at that location, and aresubject to NERC CIP requirements. The CCAs within an ESP may use routable protocols forcommunicating amongst themselves within the ESP.

For communication between systems in different ESPs, the routers in the corresponding ESPs mayuse routable protocols between them. (e.g., between routers D and P in Figure 1)

2.3 Extended Reference ArchitectureFor a general smart grid communication architecture, see [3]. In a general smart grid environment,the CCAs in the bulk electric system may need to communicate with systems in distributionsubstations or other locations as illustrated in Figure 2.

Figure 2. Extending Basic Reference Architecture to Include Other Smart Grid Elements

Smart grid systems outside of the bulk electric system are mostly not considered CCAs4. Examplesof such systems are AMI meters at customer locations, meter concentrators/collectors, IEDs(Intelligent Electronic Devices) and RTUs (Remote Terminal Units) of the distribution SCADAsystems. Please note that some of these systems may not necessarily be located at a distributionsubstation; they may be deployed at feeder or consumer locations.

As indicated above, the communication network between distinct ESPs, and between an ESP andoutside system, is not included in any ESP thus not subject to the CIP requirements of [1].

Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 3


Perimeter)



LAN

RTU

D

P


T


IED and other smartgrid elementsMeter

CollectorMeter

4 NERC CIP requirements do not address connectivity with the systems outside of the bulk electric system. However, connectivity of utilityand smart grid systems to the secure CCAs is shown here for a complete presentation of a secure utility communications architecture

3. Communication over MPLS networks

Before presenting advantages of MPLS networks towards NERC CIP compliance, communicationservices over an MPLS network are briefly described in this section. Many standards-based servicesand protocols at Layer 1, Layer 2, and Layer 3 (of the OSI framework) can be independently emulatedover the MPLS infrastructure with little impact on functionality or performance. In short, there isno need to deploy, operate and manage multiple networks for multiple functions.

3.1 MPLS ArchitectureIn an MPLS network, packets are assigned labels and transported end-to-end in logical tunnels orconnections called label switched paths (LSP). Packet forwarding decisions are based on the MPLSlabel, rather than on the protocol-specific destination information field read from the packet (suchas Ethernet MAC address, Frame Relay DLCI, IP address, etc). This (outer) label (added as partof a shim header) together with the original packet (payload) constitutes the MPLS packet (SeeFigure 3). Additional in-depth background information on MPLS architecture is given inAppendix A.

Figure 3. Packet forwarded on an LSP through an MPLS network

For purposes of this discussion, it is important to note that packets are forwarded based on thelabel, rather than the destination address as defined in the native protocol of that payload. Theend-to-end path of the LSP is pre-determined there is no change in this path as a function of thedestination address while the packet is traversing the network so the MPLS network is used tosupport non-routable protocols. Additional detail and discussion is provided in Appendix A.

Switches or routers participating in the MPLS network are called label edge routers or labelswitched routers, depending on their location in the network. Figure 3 illustrates a packet beingforwarded on an LSP through an MPLS network.

A packet is received at the ingress MPLS router (called ingress Label Edge Router ingress LER oriLER), an initial label is added and the new MPLS packet is forwarded to a determined interfaceon the LER where the LSP was configured. Label Switched Routers (LSRs) traversed by the LSPextract the label, swap with the next label and continue forwarding the packet to the egress LER(eLER), where the final label is removed.


MPLS NETWORK

LSP

LSR

Switching in the core MPLS network-Match on incoming label-Lookup outgoing label and interface-Swap labels and forward out

LSR eLER

Packet

MPLS Labels

iLER

Pkt Pkt Pkt

Pkt Pkt

LSP configuration is independent of the end to end protocols of the packets carried in the LSP orthe relationship between the end users exchanging these packets. It is possible that an LSP maycarry traffic between several sets of end users with different networking protocols. Multiplexing ofmultiple end-to-end virtual connections (VCs) with differing protocols can be achieved byproviding another (inner) label in the packet identifying the corresponding connection. Thisadditional label has only an end-to-end (LER-LER) significance and is not changed at the LSRhops. (See Figure 4).

Figure 4. Multiplexing Virtual Connections in an LSP

3.2 Converged MPLS NetworksEnd to end Layer 1 and Layer 2 communications services over MPLS are shown in Figure 5.

Figure 5. Communication Services over MPLS


MPLS NETWORK

LSR eLER

Core LSP Tunnel - Uses outer label

Psuedo wires or VCs - Use inner label Customer Nodes

iLER

Services can be defined and shared overMPLS network emulating conventionalL1, L2, and L3 connections

Independence and isolation of MPLSnetwork operations and managementfrom individual service operationsand management

Point-to-point MPLS tunnels betweenpairs of network access points

Traffic between the endpointstransparently carried in tunnel

Network supports tunnel managementincluding QoS, reliability, and security

Ethernet broadcast domain over multipleMPLS tunnel emulating- An Ethernet switch- A switched Ethernet network

Point-to-point connection through asingle MPLS tunnel emulating- PDH, SDH/SONET- (Point-to-point) Ethernet- Frame relay- ATM

VPLSVirtual Private LAN Service

Non-routable protocol

VPWS(Virtual Psuedo-wire Service)

Non-routable protocol

End-to-EndServices over

MPLS Network

3.2.1 VPWSOne of the most common implementations of MPLS-based services is Virtual Pseudo-WireService (VPWS). Also known as Virtual Leased Line (VLL) service, this service is defined byRFC 4447 [4]. A pseudo-wire is point-to-point connection between two end points at sites suchas substations, data centers, corporate sites, etc. (i.e., a tunnel).

Legacy non-routable protocols such as PDH/SONET, Frame Relay, and Ethernet can be carriedover MPLS pseudo wires. Since utilities may have numerous legacy networks in place, emulationof these connections in the converged MPLS network continue providing these services withouthaving to maintain multiple backbone infrastructures (much like a telecommunications serviceprovider does). The ability to combine multiple protocols in one network greatly simplifies andreduces the cost of operating a network.

3.2.2 VPLSAnother implementation of MPLS-based services is Virtual Private LAN Service (VPLS), as definedby RFC 4762 [5]. VPLS is a Layer 2 VPN service and is used to provide multi-point to multi-pointEthernet connectivity between substations, data centers, corporate sites, etc. and emulates anEthernet bridge connecting these endpoints. In a converged MPLS network, this Layer 2 VPN isachieved using a full mesh of LSPs between the participating sites. Conceptually, a number ofsecure tunnels are constructed, allowing multipoint connectivity.

3.3 Additional MPLS FeaturesMPLS based services provide utilities with the ability to support closed user groups (Layer 1,Layer 2 VPNs identified above) for communication among systems associated with similar applicationrequirements and/or the associated users. The isolation of traffic of one closed user group fromother groups adds to the network security and facilitates flexibility in implementing security andother requirements individually for each group. The protocols used between endpoints within aclosed user group can be different from the protocol used within another group.

With MPLS, protection mechanisms ensure that reliability requirements are met and that failurescan be recovered within specified time limits. With the combination of MPLS and DiffServ (orclass-aware traffic engineering), service differentiation can be implemented for traffic on a per-class basis (QoS classes). See the Appendices for details of some of these features.

4. Interim NERC CIP Compliance with MPLS-based Non-Routable Protocol

For some utilities, it may be difficult to implement ESP requirements since inclusion of IP routersin an ESP may be considered too costly for timely compliance of NERC CIP requirements. Withthe exemption, network connectivity from systems outside of ESP can support compliance withcommunication over non-routable protocols.

Figure 6 is an illustration of the basic architecture with the routers/switches residing outside of theESPS and communicating over non-routable protocols.


Figure 6. Example Communication Architecture with Exemption

A non-routable protocol provides Layer 1 or Layer 2 connection between two end points, or in thecase of multipoint Ethernet, connection among multiple endpoints. Even if the MPLS infrastructureis a mesh network, traffic between the two endpoints follows a pre-determined physical or logicalpath and there is no change (no routing) as a function of the protocol-specific destination information(address). Since there is no routing, packets move through their own secure tunnel. Consequently,MPLS networks provide cost effective, efficient, and flexible point-to-point and point-to-multipointconnectivity emulating a variety of non-routable protocols5. Consequently, switch/routers can belocated outside the ESP, as shown in Figure 6.

The non-routable protocols may be extended to the general smart grid network as shown in Figure 7

Figure 7. Extending Non-routable Protocols to Include Other Smart Grid Elements



Perimeter)



LAN

LAN LAN

D

C

PQ S


T


(over MPLS supporting eg,VPWS, VPLS)

Non-routable Protocol


Perimeter)



LAN

LAN LAN

D

C

PQ S


T


(over MPLS supporting eg,VPWS, VPLS)

Non-routable Protocol

and other smartgrid elementsRTU IED

MeterCollector

Meter

5 Leased line services procured from service providers are often implemented using MPLSs VPWS or VPLS service on theservice providers own MPLS networks.

5. MPLS is the Right Choice with or without the Exemption

In the near future, it is expected that the "exemption" of systems with non-routable protocols frombeing CCAs may be removed from NERC CIP requirements. In that case, all networking elementsat a BES location will be required to be in the ESP. Irrespective of whether a routable or non-routable protocol is used, MPLS technology is still the appropriate technology for communicationbetween endpoints in different ESPs, as well as between an endpoint in an ESP and other smart gridelements not in an ESP. In the case of VPWS and VPLS implementations, the tunnels inherent tothese services, and defined by the relevant standards, provide secure links between critical nodes.Should IP services be necessary, these can also be supported securely over MPLS using RFC 4364[6]. These are known as VPRN (Virtual Private Routed Network) services, or MPLS VPNs. Thesame label switching described above is utilized with a VPRN to create secure tunnels.

MPLS provides utilities with the ability to support closed user groups for communication amongsystems associated with similar application requirements and/or the associated users. Such networkseparation helps facilitate implementation of network security. MPLS protection mechanisms suchas Fast Re-route (FRR), described in Appendix B, can ensure that reliability requirements can bemet and failures recovered within specified time limits. This facility can be made available tocertain types of grid applications using class-based approaches. Furthermore, added security featuresof MPLS-based vendor products also support efficient implementation of ESP security for NERCCIP compliance.

MPLS-based converged communication networks increase operational efficiency and reducecapital expense (CAPEX). As seen in Figure 5 and 6, several types of MPLS-based services canbe implemented on a shared core. In addition to providing Layer 1 and Layer 2 connectionsusing the VPWS and VPLS technologies, L3 routable protocols (such as IP) can be implementedusing VPRNs. Utilities can use individual VPWS, VPLS, and VPRN services for independent andseparate end-to-end application connections over the same MPLS network, providing trafficsegregation for each connection. Specifically, VPWS and VPLS services are isolated not only fromVPRN IP traffic, but also Internet traffic; packets from the Internet cannot enter VPWS or VPLSservices based VPNs. This key feature allows MPLS to support network-based logical accesscontrol to protect power utility CCAs.

6. ESP Security Implementation

Removal of the non-routable exemption will result in the need to apply security to communicationsendpoints previously exempt from NERC CIP requirements. These systems will now be consideredcritical cyber assets that must be contained within an ESP. Referring to Figure 6, the electronicsecurity perimeter will now extend to include MPLS end systems C, Q and S in the Data andControl Center, Bulk Power Station, and Transmission Substation respectively. In other words,the ESP boundary will now be the interface between these switches and the power utility'scommunication network. The communication network will remain out of scope for NERC CIPrequirements (See Requirement CIP-005-B.R1.3 in [1] ), but measures must be put in place toensure that the new ESP boundary remains intact, and that the MPLS end systems comply withthose requirements.

6.1 Requirements OverviewThe nine NERC CIP requirements [1] provide auditable standards that must be in place for theprotection of bulk electric systems. CIP-001 Sabotage Reporting requires guidelines and procedures for reporting disturbances or

unusual occurrences suspected, or determined to be caused by, sabotage to the appropriatesystems, governmental agencies, and regulatory bodies, as well as informing operating personnel.

CIP-002 Critical Cyber Asset Identification requires the identification and documentation ofCritical Cyber Assets (CCAs) associated with critical assets that support the reliable operationof the bulk electric system.


CIP-003 Security Management Controls requires the implementation and documentation of acyber security program for the secure management of critical cyber assets.

CIP-004 Personnel and Training requires that personnel having authorized cyber or authorizedunescorted physical access to critical cyber assets, including contractors and service vendors,have an appropriate level of personnel risk assessment, training, and security awareness.

CIP-005 Electronic Security Perimeter(s) requires the identification and protection ofElectronic Security Perimeters (ESPs) within which all critical cyber assets reside as well as allaccess points on the perimeter.

CIP-006 Physical Security of Critical Cyber Assets mandates the implementation of a physicalsecurity program for the protection of critical cyber assets.

CIP-007 Systems Security Management requires the definition of processes, methods andprocedures for securing critical cyber assets as well as non-critical assets that are within an ESP.

CIP-008 Incident Reporting and Response Management ensures the identification,classification, response and reporting of cyber security incidents related to critical cyber assets.

CIP-009 Recovery Plans for Critical Cyber Assets requires that recovery plans are put in placefor critical cyber assets and that these plans follow established business continuity and disasterrecovery techniques and practices.

The NERC CIP-002 through NERC CIP-009 standards provide a cyber security framework for theidentification and protection of critical cyber assets supporting the reliable operation of the bulkelectric system. The majority of these standards consist of policies, procedures and best practicesthat are required to be put in place. CIP-002, CIP-005 and CIP-007 are the most technical ofthese NERC critical infrastructure protection standards. CIP-002 has been discussed at lengththroughout this paper. The technical aspects of CIP-005 and CIP-007 deserve further treatment.

6.2 ESP Identification and ProtectionCIP-005 prescribes requirements for ensuring that every critical cyber asset resides within anelectronic security perimeter. The set of CCAs within an ESP can be thought of as a securityenclave - a grouping of critical assets by function or role that can be isolated as much as possiblefrom unauthorized access. CIP-005 discusses the technical mechanisms for controlling theexternal, electronic access points to an ESP. The access points are the ports and protocols on CCAsand non-critical assets within the ESP that provide access from outside. CIP-005 also requireselectronic or manual logging of all access attempts and an annual cyber security vulnerabilityassessment of the external electronic access points to the ESP.

6.3 System Security ManagementCIP-007 is concerned with maintaining and verifying the security of CCAs and non-critical cyberassets within an ESP. CIP-007 revisits the issue of ensuring that unused ports and services aredisabled and requires the use of anti-malware software where technically feasible. CIP-007 requirestechnical controls that enforce access authentication and accountability for all user actions andcontains technical controls providing password security. Other technical control requirementsinclude controls to monitor and generate alerts for system events related to cyber security and anannual vulnerability assessment which includes a review of the controls for default accounts.

6.4 Technical Guidance for Compliance with NERC CIP RequirementsThe NERC CIP cyber security standards identify requirements that must be met to secure the bulkelectric system; they provide guidance regarding what needs to be done, but do not specify how itshould be done. The National Institute of Standards and Technology (NIST) draft standardNISTIR 7628 and the International Electrotechnical Commission (IEC) 62351 series of standardsare sources that provide technical guidance for compliance.


The second draft of NISTIR 7628, Smart Grid Cyber Security Strategy and Requirements [7] wasissued in February 2010 and describes the overall cyber security strategy for the smart grid. Itcontains technical guidance for securing the smart grid, based on a set of general purpose securityrequirements found in NIST SP 800-53 Recommended Security Controls for Federal InformationSystems and Organizations [8].

The NISTIR 7628 smart grid cyber security requirements provide comprehensive, detailed technicalguidance for securing cyber assets within an ESP and are organized according to the DHS Catalogof Control Systems Security [9] categories. The technical requirements in those categories relevantto communications systems within the bulk energy system are summarized below.

System and Communication Protection consists of steps taken to protect systems and thecommunication links between them from cyber intrusions.

System Development and Maintenance includes technical requirements that ensure secure backupsand remote maintenance.

Incident Response includes technical requirements that enable the examination of a system aswell as its recovery and/or reconstitution after a disruption or failure.

System and Information Integrity ensures that sensitive data is not modified or deleted in anunauthorized manner.

Access Control ensures that resources are only accessed by authorized personnel and that thosepersonnel are accurately identified.

Audit and Accountability ensures the existence and availability of system logs that are used todetect breaches of system security, anomaly detection, and forensic analysis.

The NISTIR 7628 Smart Grid Cyber Security Strategy and Requirements standard provides comprehensivetechnical guidance for securing critical cyber assets as well as non-critical cyber assets within anESP. The IEC 62351 series of standards (62351-1 through 62351-7) [10] is concerned with securingthe unique communication protocols used by the power utility industry. The IEC standards addressthe end-to-end security of the communication protocols, so that intervening communicationsequipment is transparent to these security standards.

7. Using ITU-Ts X.805 Security Standard to Secure the Smart Grid

Extending the ESP boundary to include previously excluded communication systems necessitates astandards-based security assessment of these systems to ensure compliance with NERC CIPrequirements and ESP security. The ITU-T X.805 [2] security standard provides a comprehensiveframework that can be used to ensure that the management, control, and end-user plane of thesesystems are secure relative to eight dimensions (access control, authentication, non-repudiation,data confidentiality, data integrity, availability, communication security, and privacy). The assessmentwill identify features within the communication systems that satisfy NERC CIP requirements. Italso identifies new security features that need to be developed or compensating controls that needto be deployed to ensure that the security of the ESP remains intact.

NISTIR 7628, Smart Grid Cyber Security Strategy and Requirements, contains security requirementsthat lead to NERC CIP compliance for cyber assets within an ESP. Each of the NISTIR 7628requirements is a control that resides in one or more X.805 security dimensions. Therefore, thecompliance level of the critical cyber asset with respect to NISTIR 7628 and NERC CIP correspondsto how well it addresses X.805 security dimensions.

Figure 8 shows how the X.805 security dimensions can be used to measure the compliance level ofthe CCA with respect to NERC CIP in an easy-to-understand pictorial manner. This format can


also be used to depict the ESP's overall compliance level with NERC CIP requirements. In orderfor the critical cyber asset to fully satisfy the NISTIR 7628 requirements, the solid green areawould have to completely cover the red area. The white area depicts the implementation level ofadditional security requirements that a power utility may want to deploy.

Figure 8. Depicting the Security Compliance Level of an Example CCA

8. Threats to the Electronic Security Perimeter

Power utilities are increasingly reliant on information technologies to manage the power grid,resulting in an integration with telecommunications networks. These networks must be managedto the same reliability as the power grid since threats to telecommunications networks nowbecome threats to the power grid.

IEC TS 62351-1, Power Systems Management and Associated Information Exchange Data andCommunications Security Part 1: Communication Network and System Security Introduction toSecurity Issues, lists the four cyber security threats to the power grid as:1. Unauthorized access to information,2. Unauthorized modification or theft of information,3. Denial of service,4. Repudiation/unaccountability.

All of these threats directly relate to the CCAs and other cyber assets within an ESP. Therefore,precautions must be taken to protect communications equipment once the ESP is extended toinclude them.

The motivations for attacking the power grid include industrial espionage, vandalism, cyberhacking and terrorism. Figure 9 indicates the principal location of threats to the bulk electricsystem. Protection against denial of service is of paramount importance to the power utilityindustry, a critical infrastructure, since the primary mission of every power utility is to provideuninterrupted electric service to its customers.


Authentication

NonRepudiation

DataConfidentiality

CommunicationSecurity

DataIntegrity

Availability

Privacy

AccessControl

77%

0%24%

31%

29% 13%68%

42%

35%35%

87%

88%

Figure 9. Threats to the Bulk Electric System

9. Potential Vulnerabilities in the ESP

Threats and threat agents will always be present in any environment and nothing can be done toeliminate them. In order to secure the power grid, vulnerabilities must be focused on. NISTIR7628 contains an extensive list of vulnerabilities that could be present in power grid equipmentand is organized into several vulnerability classes that illustrate the four threats to the ESP andhow they would be realized in the form of an attack.

Vulnerabilities in communications systems typically reside in the management or control plane asend-user traffic does not terminate on this type of equipment. Machine-to-machine interactionsoccur in the control plane, whereas both human-to-machine and machine-to-machine interactionsoccur in the management plane. An attacker can forge messages associated with any of thesecommunication types to cause denial of service, unauthorized access, unauthorized modificationor theft of information, and repudiation/unaccountability of actions.

10. Mitigations for ESP Vulnerabilities and NERC CIP Compliance

Security must be designed in at the architectural level to provide the most complete and costeffective solution. To provide defense in depth, security functionality must be implemented usinga tiered approach; an integral part of systems as well as networks. Defining power grid securityrequirements in advance will ensure that they are implemented, as retroactive deployment ofsecurity updates or compensating controls may be cost prohibitive.

Security mechanisms found in the X.805 Access Control security dimension are often the firstline of defense to mitigate communications system vulnerabilities. If an attacker cannot accessthe system, he/she cannot compromise it, take it out of operation, or exhaust its resources.Unauthorized access can be prevented in many ways, including secure user IDs and passwords,access control lists, firewalls, intrusion detection/prevention systems, etc.




LAN

LAN LAN

RTU

D

P T


(Preferably MPLS-based

IED and other smartgrid elements

Unauthorized access to information Unauthorized modification or

theft of information Denial of Service Repudiation/Unaccountability

Denial of Service Repudiation/Unaccountability

Denial of Service Repudiation/Unaccountability

MeterCollector


Perimeter)Meter

The communication network in the above architecture diagram raises concern as a potentialconduit for unauthorized access to critical cyber assets. MPLS VPNs can be used to providenetwork-based access control, as well as traffic isolation across shared or converged networks.Since they inherently provide traffic isolation capability at Layer 2, an MPLS VPN can be createdfor each security domain to deploy closed user groups as the foundation for a secured networkinfrastructure between ESPs that contribute to NERC CIP compliance.

Controls contained in the remaining X.805 security dimensions must also be deployed in order toprovide the comprehensive security required for NERC CIP compliance. After access control,availability controls are probably the most important for the power utility industry. These controlsinclude per-flow queuing, per-peer queuing, and business continuity/disaster recovery/continuity ofoperations procedures.

Attention must also be paid to authentication, data integrity, and non-repudiation securitydimensions in order to ensure that administrative actions on CCAs are performed only byauthorized personnel, who can be held accountable. This also ensures that messages transmittedbetween CCAs and across ESPs have not been forged or tampered with.

Communication security, which ensures that information travels between authorized end-pointswithout being diverted or intercepted, is also provided by MPLS VPNs. Data confidentiality andprivacy security dimensions are less important in context of the bulk electric systems, especiallythe communications system CCAs that have been focused on in this paper.

Alcatel-Lucent 77x0 SR MPLS routers offer a broad range of security features to provide the defense-in-depth necessary to protect critical cyber assets within an electronic security perimeter. SR-OSfeatures that enable secure system operation include support for SSH and SCP, support for SNMPv3,logging of system events and access requests, TCP wrappers and IP tables, session timeouts forremote management sessions, and login banners. The SR-OS also supports RADIUS and TACACS+password management, as well as password profiles for separation of duties assignments.

To protect against DoS attacks, SR-OS provides several types of ACLs, as well as CPM queues andper peer queuing. SR-OS supports CPM filters and Management Access Filters (MAFs) in additionto typical ACLs that are applied to logical ports. It also provides anti-spoofing filters, unicastReverse Path Forwarding (uRPF), Secure MAC Learning, and MAC Learning Protection toprotect against spoofed IP and MAC addresses. The uRPF feature discards packets with unverifiableIP source addresses, and Secure MAC Learning prevents the registration of another MAC addresson a VPLS service access point (SAP) or service distribution point (SDP) after one is initiallyregistered. MAC Learning Protection discards MAC update requests for protected addressesthat originate from unprotected service access points.

The combination of DHCP Snooping and SR-OS provided ARP Reply Agent features can be usedto protect against DHCP starvation and ARP spoofing attacks. Using the ARP Reply Agent toreceive and generate requests from subscribers can restrict the MAC-IP pairs registering on thenetwork. The ARP Reply Agent uses DHCP Snooping to listen to the DHCP exchange andpopulate its table of known IP, MAC address pairs. When new ARP requests are attempted, theyare compared to known values to prevent an attacker from spoofing an ARP request or response.

SR-OS provides MD5 hashes for supported routing protocols, as well as BGP TTL protection toprotect the integrity of routing protocol messages. BGP TTL protection leverages the fact that theTTL field of received BGP updates should never be less than 254 since they always originate froma neighboring router.


11. Conclusions

This paper establishes that MPLS based networks provide secure, reliable, efficient, flexible andcost-effective communication between the Critical Cyber Assets at different locations of the BulkElectric System, as well as between CCAs and other smart grid network elements.

We have shown that MPLS networks natively support communication with non-routable protocols.With the current requirement exemption, Bulk Electric System endpoints connecting to theMPLS networks that are not currently considered to be CCAs, and allowed to be outside the ESPboundary, can communicate over the MPLS network with non-routable protocol.

It is further shown that, if necessary, MPLS networks can be extended for secure communicationbetween CCAs of the bulk electric systems and other smart grid network elements in compliancewith NERC CIP. The traffic isolation capabilities inherent in MPLS-based services providenetwork-based access control for bulk energy system CCAs and other smart grid network elements.Several types of MPLS services can be configured to establish closed user groups. IP packetsoriginating from external endpoints, including the Internet, cannot enter these closed user groups,thus preventing many types of external attacks.

If the non-routable protocol exemption is removed, all communication endpoints providing externalaccess into an ESP, including MPLS endpoints, will be considered critical cyber assets andtherefore subject to NERC CIP compliance. This paper has provided guidance for securing thesecommunication endpoints and pointed out several security features present in Alcatel-LucentMPLS switches that comply with the NERC CIP requirements.


References

[1] North American Electric Reliability Corporation, Reliability Standards for the Bulk ElectricSystems of North America, NERC, December 2009.www.nerc.com/files/Reliability_Standards_Complete_Set_2009Nov2.pdf

[2] International Telecommunication Union Telecommunication Standardization Sector (ITU-T),Security Architecture for Systems Providing End-to-End Communications, ITU-T Rec. X.805,October 2003.//www.itu.int/rec/T-REC-X.805-200310-I

[3] K. C. Budka, J. G. Deshpande, T. L. Doumi, M. Madden, T. Mew, Communication NetworkArchitecture and Design Principles, to appear in Bell Labs Technical Journal special issue onEco-Sustainability and Green Information and Communication Technology, Summer 2010.

[4] L. Martini, Ed., E. Rosen, T. Smith, G. Heron Pseudo-wire Setup and Maintenance Using theLabel Distribution Protocol (LDP), IETF RFC 4447, April 2006.

[5] M. Lassers, Ed., V. Kampala, Ed., Virtual Private LAN Service (VPLS) Using Label DistributionProtocol (LDP) Signaling IETF RFC 4762, January 2007.

[6] E. Rosen, Y. Rekhter BGP/MPLS IP Virtual Private Networks (VPNs), IETF RFC 4363February 2006.

[7] National Institute of Standards and Technology, Smart Grid Cyber Security Strategy andRequirements, DRAFT NISTIR 7628, February, 2010.csrc.nist.gov/publications/drafts/nistir-7628/draft-nistir-7628.pdf

[8] National Institute of Standards and Technology, Recommended Security Controls for FederalInformation System and Organizations, NIST SP 800-53, Revision 3, August 2009.csrc.nist.gov/publications/nistpubs/800-53-Rev3/sp800-53-rev3-final-errata.pdf

[9] Department of Homeland Security, National Cyber Security Division, Catalog of Control SystemsSecurity: Recommendations for Standards Developers, September 2009.www.us-cert.gov/control_systems/pdf/Catalog_of_Control_Systems_Security_Recommendations.pdf

[10] International Electrotechnical Commission, Power Systems Management and AssociatedInformation Exchange Data and Communications Security(all parts), IEC TS 62351-1 62351-7,2005 2009.www.iec.ch

[11] L. Anderson, T. Madsen, Provider Provisioned Virtual Private Network (VPN) Terminology, IETFRFC 4026, March 2005.

[12] P. Pan, G. Swallow, A. Atlas, Fast Reroute Extensions to RSVP-TE for LSP Tunnels, IETF RFC4090, May 2005.


AcronymsACL Access Control List

AMI Advanced Metering Infrastructure

ARP Address Resolution Protocol

ATM Asynchronous Transfer Mode

BES Bulk Electric System

BGP Border Gateway Protocol

CCA Critical Cyber Asset

CES Circuit Emulation Service

CIP Critical Infrastructure Protection

CPM Control Processor Module

DER Distributed Energy Resource

DHCP Dynamic Host Configuration Protocol

DNP3 Distributed Network Protocol version 3

DoS Denial of Service

ESP Electronic Security Perimeter

FR Frame Relay

FRR Fast Re-Route

FTP File Transfer Protocol

ICCP Inter-Control Center Communications Protocol

IDS Intrusion Detection System

IEC International Electrotechnical Commission

IED Intelligent Electronic Device

IETF Internet Engineering Task Force

IP Internet Protocol

IPS Intrusion Prevention System

IPSec IP Security protocol

ITU-T International Telecommunication UnionTelecommunication Standardization Sector

LAN Local Area Network

LER Label Edge Router

LSP Label Switched Path

LSR Label Switching Router

MAC Media Access Control

MAF Management Access Filter

MD5 Message Digest version 5

NAT Network Address Translation

NIST National Institute of Standards and Technology

NERC North American Electric Reliability Corporation

OSI Open Systems Interconnection

PDH Plesiochronous Digital Hierarchy

PKI Public Key Infrastructure

RADIUS Remote Access Dial-In User Service Protocol

RFC Request For Comments

RTU Remote Terminal Unit

SAP Service Access Point

SCADA Supervisory Control and Data Acquisition

SCP Secure Copy

SDP Service Distribution Point

SHA-1 Secure Hash Algorithm version 1

SNMPv3 Simple Network Management Protocol version 3

SONET Synchronous Optical Network

SSH Secure Shell

SSL Secure Sockets Layer

TACACS+ Terminal Access Controller Access ControlSystem Plus Protocol

TASE.2 Telecontrol Application Service Element version 2

TCP Transmission Control Protocol

TDM Time Division Multiplexed

TLS Transport Layer Security

TTL Time To Live

uRPF unicast Reverse Path Forwarding

VC Virtual Channel

VLAN Virtual Local Area Network

VLL Virtual Leased Line

VoIP Voice over IP

VPLS Virtual Private LAN Service

VPN Virtual Private Network

VPWS Virtual Pseudo-Wire Service


Appendix A. MPLS Architectures

In an MPLS network, packets are assigned labels and transported end-to-end in logical tunnels orconnections called label switched paths (LSP). Packet forwarding decisions are made based on theMPLS label rather than on the contents of the destination address field from the packet. The labelis added as part of a shim header of a packet considered the payload of the MPLS packet. Anillustration of the MPLS shim header is shown in Figure 10.

Figure 10. High level MPLS packet structure showing the shim header

On the right hand side of the figure is the MPLS payload (eg, SONET, Ethernet, ATM or FrameRelay frame, IP packet). The MPLS payload contains its native protocol header and its own payload,in this case corresponding to the data being transmitted. On the left hand side of the figure is theMPLS shim header that includes a 20-bit label.

Packets are forwarded based on the label, rather than routed based on the destination addressdefined in the native protocol of that payload. The end-to-end path of the LSP is pre-determinedand there is no change in this path as a function of the destination address while the packet istraversing the network. Thus, MPLS networks support non-routable protocols.

LSPs are uni-directional and established end-to-end; the labels associated with an LSP areknown to each incident switch or router involved in its establishment. The switches or routersparticipating in the MPLS network are called label edge routers or label switched routers, dependingon their location in the network. Figure 3 shows an illustration of a packet being forwarded on anLSP through an MPLS network.

When a packet is received at the ingress MPLS router (called a Label Edge Router LER), an initiallabel is added and the new MPLS packet is forwarded to a determined interface on the LER wherethe LSP was configured. An appropriate interface was identified when the LSP was created. TheLabel Switched Routers (LSRs) traversed by the LSP extract the label, swap with the next label andcontinue forwarding the packet to the egress LER where the final label is removed. The LSPconfiguration is independent of the end-to-end protocols of the packets carried in the LSP or therelationship between the end users exchanging these packets. It is possible that an LSP may carrytraffic between several sets of end users with different networking protocols. MPLS providesanother important capability called label stacking. More than one MPLS shim header can be added


PayloadMPLS Shim Header

DataProtocolHeader

Label(20 bits)

OtherFields

Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper18

to the packet, thereby stacking labels. This provides two important facilities. The first is theconcept of a pseudo wire or virtual channel [11], which allows different service types (e.g.different sets of end users using different protocols) to be carried over a common LSP in the MPLScore while maintaining separation between them. The second is the concept of service protection,which we discuss in Appendix B. Each traffic stream is confined to its own pseudo wire or VC[11], as identified by its VC identifier. The label stacking capability of MPLS can then be used toembed the VC identifier as a label in the MPLS packet. The VC label is often called the inner label[11] and does not change as the packet traverses through the LSP. The outer label corresponds tothe LSP in the core network (which we referred to as the MPLS label earlier), and its value maychange as the packets pass through the LSRs


Appendix B. Additional MPLS Features

In this section, we describe additional MPLS features that can be deployed to ensure reliabilityrequirements can be met through protection mechanisms, and allow service differentiation to beimplemented for traffic on a per-class basis.

MPLS ProtectionFor an LSP tunnel that is established between two endpoints or LERs (at the substations or sites),two types of protection mechanisms can be employed (individually or in combination). The first isthe use of a backup or standby LSP that uses a secondary path that is a link and/or node disjointfrom the first one. This ensures that if there is a single link or node failure in the core network thataffects the LSP, a secondary LSP is available as a backup path to carry the data. Since the secondaryLSP also consumes bandwidth, this type of protection is typically used for traffic with very stringentprotection requirements. In the following figure, we show an example of a backup/standby LSP.

Figure 11. Primary LSP and backup/standby LSP with disjoint secondary path

The primary and secondary LSPs are shown in Figure 11. Secondary LSPs can also be established inthe network deployment phase along with the primary LSPs and no new LSP provisioning needs tobe performed after a failure has occurred. This also maintains the non-routable nature of the network.

The second type of available protection is referred to as fast reroute (FRR) or facility bypass (RFC[12]). FRR uses the label encapsulation method to bypass single link or node failures in the network.However, when FRR is active, there may be temporary congestion in parts of the network wherethe bypass has been established. The following figure shows an example where bypass tunnels areestablished to protect against node failures.

Figure 12. Primary LSP and FRR bypass tunnels to protect against router failures


LSRLSR

LSR

LSR

eLER

LSP- Secondary Path

LSP- Primary Path

iLER

IPIP

LSR LSR LSR

R2R3

LSR

R4

eLER

Bypass tunnel toprotect R2 failure

Protected (Primary) LSP

iLER

R1

R6

R7 R9R8

R5

Bypass tunnel toprotect R4 failureBypass tunnel to

protect R3 failure

In the figure above, the primary LSP is shown as a solid red line. The LSPs shown with dotted linesrepresent bypass tunnels used to forward traffic around failed routers. Note that the protectionmechanism consists of stacking a new label on a packet and forwarding along the bypass tunnel.Thus, SONET-like protection times in the range of 50 ms. can be achieved. The FRR tunnels canbe designed and deployed along with the primary LSPs, again maintaining the non-routable natureof the network.

MPLS Class-Based Quality of Service DifferentiationDiffServ or class-aware MPLS implementations allow differentiation of quality of service (QoS) ona per-class basis. Service differentiation can be performed by assigning packets of a particularapplication to a specific traffic class. In the MPLS core network, LSPs can be set up to carry packetsof one or more specific class types. At each of the routers in the core network, bandwidth allocationand priorities (hence, the treatment of the packets) can be set according to the different classes.This allows great flexibility in how applications are delivered in the network. For example, videosurveillance applications can be allocated specific bandwidth, but policed to make sure the bandwidthused does not starve other applications; VoIP traffic can be assigned to low latency paths; networkcontrol data and incident-related communications can be assigned highest priority, etc.

While many application types exist, the number of traffic classes is technically limited to 8; thesetypically represent 4 classes with 2 priority levels each. When the converged utility network is setup, the allocation of application types to the traffic or QoS classes must be performed. In this mapping,applications sharing similar characteristics would typically be assigned to the same class type.

In addition to class-based differentiation, MPLS networks also incorporate the concept of trafficengineering. Unlike traditional IP networks, where traffic takes simple shortest path routes, MPLStraffic engineering allows LSP paths to be established that are optimal according to criteria otherthan the shortest path and can take into account the available bandwidth on individual links.Thus, traffic engineering can result in more efficient use of bandwidth resources in IP network, andhelps to further reduce the overall cost of operations. When class-aware traffic engineering isimplemented, high priority/low latency traffic can be carried on shorter, less congested paths toensure the QoS criteria are being met. The following figure shows an example.

Figure 13. Two LSPs established from R2 to R5 using MPLS class-aware traffic engineering

In Figure 13, high priority critical data from R2 to R5 (e.g. control data) is sent on the high priority(red with solid line) LSP, whereas non-critical operations data is sent on the low priority (blue withdotted line) LSP. Class-based traffic engineering is very important in meeting the QoS requirementsof a diverse set of applications.


R2R3

R4Low Priority LSP

R1

R6

R7 R9R8

R5

High Priority LSP

Appendix C. Technical Guidance for Compliance with NERC CIP Requirements

NERC CIP requirements provide a comprehensive set of best practices that must be followed tosecure the bulk electric system. They provide guidance regarding what needs to be done, howeverthey do not specify how it should be done. The National Institute of Standards and Technology(NIST) draft standard NISTIR 7628 and the International Electrotechnical Commission (IEC)62351 series of standards are sources that provide technical guidance for compliance with theNERC CIP requirements.

The second draft of NISTIR 7628, Smart Grid Cyber Security Strategy and Requirements [7] was issuedin February 2010 and describes the overall cyber security strategy for the smart grid. It also containstechnical guidance for securing the smart grid that is based on a set of general purpose securityrequirements found in NIST SP 800-53 Recommended Security Controls for Federal InformationSystems and Organizations [8].

The NISTIR 7628 smart grid cyber security requirements provide comprehensive, detailed technicalguidance for securing cyber assets within an ESP and are organized according to the DHS Catalogof Control Systems Security [9] categories. The technical requirements in those categories relevantto communications systems within the bulk energy system are summarized below.

System and Communication Protection consists of steps taken to protect systems and thecommunication links between them from cyber intrusions. This category includes technicalrequirements involving: partitioning management traffic, isolating security functions, preventing DoS attacks, prioritizing the use of system resources, protecting the authenticity, integrity and confidentiality of communicated information, establishing and providing trusted communications paths,Managing cryptographic keys, using validated cryptographic algorithms, using PKI certificates, identifying and protecting external communications connections, establishing security roles for all users.

System Development and Maintenance. This category includes technical requirements involving: performing secure backups of critical software, applications and data, authorizing, managing and monitoring remote maintenance.

Incident Response. This category includes technical requirements involving: system operation in a safe/limited mode that allows the examination of logs and configuration

information, as well as resetting, enabling and disabling the system,mechanisms to enable recovery and/or reconstitution of the system by authorized personnel

after a disruption or failure.


System and Information Integrity ensures that sensitive data is not modified or deleted in anunauthorized manner. The technical requirements in this category involve: system and information integrity procedures, protection against malicious code, network intrusion detection systems, access control lists (ACLs), dynamic packet filtering, system hardening, logging and reporting of security events and system activities, detection of unauthorized changes to software and information.

Access Control ensures that resources are only accessed by authorized personnel and that thosepersonnel are accurately identified. The technical requirements in this category involve: access control policies and procedures for management tasks that are commensurate with the

criticality of the task, identification and authentication controls, including the authentication of communications

between systems, account management including establishment, activation, modification, reviewing, disabling

and removing accounts (e.g., default accounts and passwords, temporary accounts, emergencyaccounts, inactive accounts),

auditing account management activities,managing user identifiers and authenticators: associating a unique identifier with each user or

process, disabling unused identifiers, authorizing user identifiers, support for individual, role-based, group-based, and device-based user identification

and authentication, authenticator management: defining authenticator content, distributing authenticators,

periodic changing of authenticators, changing default authenticators, enforcing assigned authorizations for controlling access to the system, restricting access to

privileged functions and security-relevant information to authorized personnel, access control support for separation of duties and least privilege, prohibition of anonymous, guest and public accounts, obfuscation of authentication input (e.g., displaying asterisks when the password is typed), controlling the flow of information between interconnected systems in accordance with

applicable policy, password security including password complexity, password expiration, acceptable system use notification, limiting the number of concurrent sessions for a user, notifying a user, after successful logon, of the date and time of the last successful logon and

the number of intervening unsuccessful logons, limiting the number of unsuccessful logon attempts, locking sessions (remote and local) after a period of inactivity, securing remote access, preventing access to the system from the operator's enterprise network, recording and reporting unauthorized access attempts to the system.


Audit and Accountability ensures the existence and availability of system logs that are used todetect breaches of system security, anomaly detection, and forensic analysis. The technicalrequirements in this category involve: generation of audit records for security events, control events, configuration changes, transmission of audit records and logs to a centralized log management system for long-term

storage and correlation, content of audit records, local storage capacity for audit records on the system, alerting when the audit system fails (e.g., the audit storage utilization exceeds a previously

defined percentage of audit storage capacity), automated detection and alerting mechanisms for inappropriate, unusual or suspicious activity

or security violations, providing time stamps in audit log records, protection of audit log information and tools from unauthorized access.

The NISTIR 7628 Smart Grid Cyber Security Strategy and Requirement standard provides comprehensivetechnical guidance for securing critical cyber assets as well as non-critical cyber assets within an ESP.The IEC 62351 series of standards (62351-1 through 62351-7) [10] is concerned with securing theunique communication protocols used by the power utility industry. These protocols are: IEC 60870-5 is widely used in Europe and other non-US countries for SCADA system to RTU

data communications. It is used both in serial links (IEC 60870-5-101) and TCP/IP networks(IEC 60870-5-104). DNP3 was derived from IEC 60870-5 for use in the USA and is now widelyused in many other countries as well, primarily for SCADA system to RTU data communications.

IEC 60870-6 (also known as TASE.2 or ICCP) is used internationally for communicationsbetween control centers and often for communications between SCADA systems and otherengineering systems within control centers.

IEC 61850 is used for protective relaying, substation automation, distribution automation,power quality, distributed energy resources (DERs), substation to control center, and otherpower industry operational functions. It includes profiles to meet the ultra fast response times ofprotective relaying and for the sampling of measured values, as well as profiles focused on themonitoring and control of substation and field equipment.

IEC 62351 series of standards addresses the end-to-end security of these protocols, as such, theintervening communications equipment is transparent to these security standards. Modbus, Fieldbusand proprietary communication protocols are used by legacy systems. These protocols typicallyprovide serial communication between RTUs and SCADA systems and must also be protectedend-to-end.


Appendix D. The X.805 Security Dimensions

The eight X.805 security dimensions provide a framework to understand the types of measures thatneed to be applied to the management plane, control plane, and end-user plane in order to protectcyber assets within an ESP. (See [2].)

1. The Access Control dimension ensures that only authorized personnel or devices are allowed toaccess the communications system. Example access control measures that can be put in place tomaintain an ESP include ACLs, firewalls, IDS/IPS, access filters, user profiles.

2. The Authentication dimension confirms the identities of communicating entities. Exampleauthentication measures that can be put in place to maintain an ESP include passwords,two-factor authentication, anti-spoofing, digital signatures, digital certificates.

3. The Non-Repudiation dimension prevents deniability of an action or activity. Example non-repudiation measures that can be put in place to maintain an ESP include logging anddigital signatures.

4. The Data Confidentiality dimension protects information from unauthorized disclosure; it ensuresthat data content cannot be understood by unauthorized individuals. Data encryption and filesystem/database access controls are two example data confidentiality measures that can be putin place to maintain an ESP.

5. The Communication Security dimension ensures that information flows between authorizedend-points without being diverted or intercepted. Example communication security measuresthat can be put in place to maintain an ESP include tunneling protocols, IPsec, VLANs, VPNs,SSH, SSL/TLS.

6. The Data Integrity dimension protects against unauthorized modification, creation, deletionand replication of data that is in-transit or at rest. It also provides an indication of theseunauthorized activities. Example data integrity measures that can be put in place to maintain anESP include MD5 and SHA-1 hashes, message authentication codes, message digests, anti-virus/anti-malware software.

7. The Availability dimension ensures that there is no denial of authorized access to networkresources. Example availability measures that can be put in place to maintain an ESP includepacket filtering, per-flow queuing, per-peer queuing, business continuity/disasterrecovery/continuity of operations procedures.

8. The Privacy dimension provides for the protection of information that might be derived fromthe observation of network activities (e.g., traffic analysis). Example privacy measures thatcan be put in place to maintain an ESP include private IP addresses, NAT, web proxies, webanonymizer services.


Appendix E. Potential Vulnerabilities in the Power Grid

NISTIR 7628 contains an extensive list of vulnerabilities that could be present in power gridequipment. The vulnerability list is organized into several vulnerability classes that would allow thefour threats to the ESP to be realized in the form of an attack. Since the focus of this paper is theeffect of extending the electronic security perimeter to include previously excluded communicationsequipment, we will only consider vulnerability classes applicable to the security functionalcapabilities of communication systems.

Code quality vulnerabilities that allow an attacker to stress the system in unexpected ways. Authentication bypass or other circumvention/manipulation of the authentication process. Authorization vulnerabilities that allow authenticated entities to perform actions the security

policy does not allow. Cryptographic vulnerabilities that allow an attacker to view, modify or forge encrypted data as

well as impersonate another party by using compromised digital signatures. Logging and auditing vulnerabilities that aid an attack or increase its likelihood of success by

allowing the attacker to cover his/her tracks. Password management vulnerabilities that allow an attacker to obtain or guess passwords. Use of insecure protocols for which security was not sufficiently considered during the

development process (e.g., telnet, ftp). Installed security capabilities not enabled by default. Un-needed services running. Insufficient log management, inadequate security monitoring and event logging, no centralized

log server. Inadequate integrity checking of messages; the integrity of protocol messages and data messages

should be verified before routing or processing. Inadequate network segregation to control traffic between security zones. Weakness in authentication process or in authentication keys; the authentication mechanism

does not sufficiently authenticate devices or exposes authentication keys to attack.


www.alcatel-lucent.com Alcatel, Lucent, Alcatel-Lucent and the Alcatel-Lucent logoare trademarks of Alcatel-Lucent. All other trademarks are the property of their respective owners.The information presented is subject to change without notice. Alcatel-Lucent assumes no responsibilityfor inaccuracies contained herein. Copyright 2010 Alcatel-Lucent. All rights reserved.EPG1806100705 (07)

WhitePaper_MPLS and NERC CIP Compliance BellLab

Documents

Transcript of WhitePaper_MPLS and NERC CIP Compliance BellLab