Enter Report Title Here - UCAIugosgug.ucaiug.org/utilisec/Shared Documents/EPRI... · Web viewHow...

Advanced Metering Infrastructure Incident Response

Final Report

Electric Power Research Institute

August 10, 2012

This document contains three separate EPRI reports that have been combined into a single document for convenience of review:

Advanced Metering Infrastructure Common Alarms and Events

Intrusion Detection System for Advanced Metering Infrastructures

Advanced Metering Infrastructure Cyber Security Incident Response Guidelines

1

2

3

4

5

6

7

89

10

11

12

13

14

15

16

17

Advanced Metering Infrastructure Common Alarms and Events

Product ID Number

18

19

20

21

22

23

24

Advanced Metering Infrastructure Common Alerts and Alarms

Product ID Number

Draft Release – August 10, 2012

Insert appropriate EPRI Title Page Auto Text entry here.

25

26

27

28

29

30

31

32

33

34

35

1

DISCLAIMER OF WARRANTIES AND LIMITATION OF LIABILITIESTHIS DOCUMENT WAS PREPARED BY THE ORGANIZATION(S) NAMED BELOW AS AN ACCOUNT OF WORK SPONSORED OR COSPONSORED BY THE ELECTRIC POWER RESEARCH INSTITUTE, INC. (EPRI). NEITHER EPRI, ANY MEMBER OF EPRI, ANY COSPONSOR, THE ORGANIZATION(S) BELOW, NOR ANY PERSON ACTING ON BEHALF OF ANY OF THEM:

(A) MAKES ANY WARRANTY OR REPRESENTATION WHATSOEVER, EXPRESS OR IMPLIED, (I) WITH RESPECT TO THE USE OF ANY INFORMATION, APPARATUS, METHOD, PROCESS, OR SIMILAR ITEM DISCLOSED IN THIS DOCUMENT, INCLUDING MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, OR (II) THAT SUCH USE DOES NOT INFRINGE ON OR INTERFERE WITH PRIVATELY OWNED RIGHTS, INCLUDING ANY PARTY'S INTELLECTUAL PROPERTY, OR (III) THAT THIS DOCUMENT IS SUITABLE TO ANY PARTICULAR USER'S CIRCUMSTANCE; OR

(B) ASSUMES RESPONSIBILITY FOR ANY DAMAGES OR OTHER LIABILITY WHATSOEVER (INCLUDING ANY CONSEQUENTIAL DAMAGES, EVEN IF EPRI OR ANY EPRI REPRESENTATIVE HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES) RESULTING FROM YOUR SELECTION OR USE OF THIS DOCUMENT OR ANY INFORMATION, APPARATUS, METHOD, PROCESS, OR SIMILAR ITEM DISCLOSED IN THIS DOCUMENT.

Reference herein to any specific commercial product, process, or service by its trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by EPRI.

The following organization(s), under contract to EPRI, prepared this report:

Southwest Research Institute

This is an EPRI Technical Update report. A Technical Update report is intended as an informal report of continuing research, a meeting, or a topical study. It is not a final EPRI technical report.

NOTEFor further information about EPRI, call the EPRI Customer Assistance Center at 800.313.3774 or e-mail [email protected].

Electric Power Research Institute, EPRI, and TOGETHERSHAPING THE FUTURE OF ELECTRICITY are registered service marks of the Electric Power Research Institute, Inc.

Copyright © 2011 Electric Power Research Institute, Inc. All rights reserved.

2

36

37383940

414243444546

4748495051

5253

54555657585960

6162636465

66

6768

6970

71

3

ACKNOWLEDGMENTSThe following organization(s), under contract to the Electric Power Research Institute (EPRI), prepared this report:

Southwest Research Institute6220 Culebra RoadP.O. Drawer 28510San Antonio, Texas 78228-0510

Principal InvestigatorT. Do

This report describes research sponsored by Electric Power Research Institute (EPRI).

This publication is a corporate document that should be cited in the literature in the following manner:

Title of Document: Subtitle. EPRI, Palo Alto, CA: <Year>. <Product ID#>.iii

72

7374

75767778

7980

81

45

6

7

ABSTRACTIn order to identify a common set of Advanced Metering Infrastructure (AMI) electric meter alarms and events for standardization, it is important to identify which alarms and events are most critical and valuable for detecting and responding to AMI security incidents. This document contains the results of Common AMI Alarms and Events Task, which is a component of the Electric Power Research Institute's (EPRI) AMI Incident Response Project. The document provides information that can be referenced in order to develop a standard for AMI alarms and events. This standard will help meet electric utilities’ need for greater interoperability and standardization in AMI electric meter alarms and events and enable vendors of Security Information and Event Management (SIEM) to provide better situational awareness and cyber event detection in their products.

v

82

83848586878889909192

8

CONTENTS1 INTRODUCTION..................................................................................................................1-1

1.1 Purpose and Scope..................................................................................................1-11.2 EPRI AMI Incident Response Project.......................................................................1-11.3 Document Development Process.............................................................................1-21.4 Document Organization............................................................................................1-3

2 OPERATING ENVIRONMENT.............................................................................................2-42.1 AMI Security Threats and Objectives........................................................................2-5

3 CATEGORIES OF ALARMS AND EVENTS.......................................................................3-63.1 Authentication...........................................................................................................3-7

3.1.1 C12.XX................................................................................................................3-83.2 Integrity.....................................................................................................................3-9

3.2.1 Event Log Storage and Management.................................................................3-93.2.2 Usage Data.......................................................................................................3-10

3.3 Controls...................................................................................................................3-103.3.1 Meter Disconnect Switch..................................................................................3-10

3.4 Anomaly Detection Services...................................................................................3-113.4.1 Metrology..........................................................................................................3-113.4.2 Firmware/Software............................................................................................3-12

3.5 Cryptographic Services...........................................................................................3-123.5.1 Key Management and Certificates....................................................................3-12

3.6 Notification and Signaling Services.........................................................................3-133.6.1 Communications Interfaces..............................................................................3-133.6.2 System Security Alarms and Events.................................................................3-143.6.3 Device Alarms and Events................................................................................3-14

4 COMMON SECURITY OBJECT DEVELOPMENT PLAN.................................................4-164.1 Attributes.................................................................................................................4-164.2 Retention Capabilities.............................................................................................4-174.3 Development Roadmap..........................................................................................4-18

5 CONCLUSION AND NEXT STEPS...................................................................................5-196 APPENDIX: REFERENCES, GLOSSARIES, AND INDEXES..........................................6-20

6.1 References..............................................................................................................6-206.2 Acronyms................................................................................................................6-20

7 APPENDIX: MALICIOUS AMI SECURITY EVENT SCENARIOS.....................................7-227.1 Scenario 1: Simple Physical Meter Attack..............................................................7-22

7.1.1 Attack................................................................................................................7-227.2 Scenario 2: Complex Cyber-Physical Meter Attack................................................7-23

7.2.1 Attack Stage 1: Physical Removal of Meter......................................................7-237.2.2 Attack Stage 2: Physical Attack of Meter Hardware.........................................7-237.2.3 Attack Stage 3: Usage of Stolen Credentials....................................................7-24

vii

9

93

9495969798

99100

101102103104105106107108109110111112113114115116117

118119120121

122

123124125

126127128129130131132

133

10

LIST OF FIGURESFigure 1 Common Alarms and Events Document Development Process...........................................1-3Figure 2 AMI Network Diagram (Courtesy Justin Searle, UtiliSec)...................................................2-4Figure 3 Simple Physical Meter Attacks...................................................................................7-22Figure 4 Complex Attack Stage 1.............................................................................................7-23Figure 5 Complex Attack Stage 2.............................................................................................7-24Figure 6 Complex Attack Stage 3.............................................................................................7-25

ix

11

134

135136137138139140141

142

12

LIST OF TABLESTable 1 C12.XX Alarms, and Events..........................................................................................3-8Table 2 Event Log Storage and Management Alarms and Events............................................3-9Table 4 Usage Data Alarms and Events..................................................................................3-10Table 5 Meter Disconnect Switch Alarms and Events..............................................................3-11Table 6 Metrology Alarms and Status Events..........................................................................3-11Table 7 Software Alarms, and Events......................................................................................3-12Table 8 Key Management Alarms and Status Events..............................................................3-13Table 9 Communication Interface Specific Events...................................................................3-13Table 10 System Security Alarms and Events.........................................................................3-14Table 11 Physical and Device Alarms and Events...................................................................3-14Table 12 Attributes...................................................................................................................4-16

xi

13

143

144145146147148149150151152153154

155

14

1INTRODUCTION1.1 Purpose and ScopeThe Advanced Metering Infrastructure (AMI) Common Alarms and Events document is intended to provide a foundation on which AMI vendors and asset owners can develop a standard for AMI alarms and events. The standard will address electric utilities’ need for greater interoperability and standardization of security events and enable Security Information and Event Management (SIEM) vendors to provide better situational awareness and cyber event detection in their products.

The scope of this document includes only alarms and events generated by the meter and not the supporting AMI components such as the collection engine, technician service tool, or other AMI equipment at the head end. Additionally, while this document attempts to begin codifying the meaning of alarms and events, it does not attempt to codify their interpretation.

1.2 EPRI AMI Incident Response ProjectThe goal of the EPRI AMI Incident Response Project is to address several challenges that confront utilities when they are designing systems for detecting and responding to AMI incidents:

Lack of a standard set of alarms and events across AMI vendors

How to design a scalable Intrusion Detection System (IDS)

Guidelines for responding to AMI incidents

If a utility deploys meters from multiple vendors, the different data formats and naming conventions for alarms and events can make it challenging and expensive to integrate the AMI systems into the utility's SIEM. A set of common security information objects is needed to standardize the AMI alarms and events across vendors. Codifying security information objects will reduce the resources necessary for a utility to deploy multiple AMI vendors and also reduce the customization that each AMI vendor must do for a specific utility. In the long run, this will also make it easier for third-party security appliance manufacturers to apply their technology to the AMI domain, helping utilities better manage their security.

The effective design of an IDS in a utility's AMI environment has several characteristics that differentiate it from the traditional Information Technology (IT) environment. For example, simply deploying a perimeter IDS may not provide the coverage necessary for an AMI system. Since there tends to be mesh networks in addition to IP-based backhaul networks, positioning an IDS at the AMI head-end system could miss malicious activity in the mesh network. Additionally, there can be scalability issues as some utilities deploy millions of meters in their service territory.

In addition to the challenges of integrating AMI systems into utility backend systems, it is also unclear what the best practices are for responding to AMI incidents. There is a need for clear guidelines for determining the type of incident (natural or manmade, malicious or non-malicious) as well as the best approach for responding to the incident.

1

15

156

157

158

159160161162163164

165166167168

169

170171172

173

174

175

176177178179180181182183

184185186187188189190

191192193194

16

This report describes the results of the AMI Alarms and Events Task that was created to identify a set of AMI alarms and events that can be standardized and develop a plan for the standardization of common AMI security objects. The other AMI incident response efforts are summarized in a separate report.

1.3 Document Development ProcessThis document is informed by and builds upon documents such as the Security Profile for AMI v2.0, AMI System Security Requirements v1.01, NISTIR 7628, and the Smart Grid (SG) Network System Requirements Specification.

The Security Profile for AMI represents the security concerns of the Advanced Security Acceleration Project for the Smart Grid (ASAP-SG) AMI Security (AMI-SEC) Task Force and provides guidance and security controls to organizations developing or implementing AMI solutions. The intent of the document is to provide prescriptive, actionable guidance on how to build-in and implement security for AMI smart grid functionality. The scope of the Security Profile for AMI extends from the Meter Data Management System (MDMS) up to and including the Home Area Network (HAN) interface of the smart meter.

AMI System Security Requirements, also a product of the AMI-SEC Task Force, provide the utility industry and vendors with a set of security requirements for AMI. The requirements are intended to be used in the procurement process, and represent a superset of requirements gathered from current cross-industry accepted security standards and best practice guidance documents.

The NISTIR 7628, Guidelines for Smart Grid Cyber Security was developed by the Cyber Security Working Group (CSWG) of the Smart Grid Interoperability Panel (SGIP), a public-private partnership launched by NIST. The NISTIR 7628 was created to provide organizations with an analytical framework to develop effective cyber security strategies tailored to the implementation of Smart Grid and their associated risks and vulnerabilities.

The SG Network System Requirements Specification was created to provide utilities, vendors, and standard development organizations with a system requirements specification for smart grid communication.

The process followed in the development of this document is depicted in Figure 1. First, AMI vendors were contacted to build support and participation. These parties helped develop a questionnaire that was then distributed to the smart grid community, including groups such as the National Institute of Standards and Technology (NIST) Smart Grid Interoperability Panel Cyber Security Working Group (SGIP–CSWG) AMI Security Group and the National Electric Sector Cybersecurity Organization Resources (NESCOR). Members of the community then formed a small informal task force to review and provide input into the creation of this document.

The task force’s charter consisted of providing information and insight from industry stakeholders as input to this document. The task force reviewed responses to the questionnaire and provided additional content. Task force members then converted the questionnaire responses into the content presented in this document.

Each draft version of this document will be distributed to the OpenSG AMI Security Group and NESCOR for feedback. This feedback will be considered by the WG leading to new versions of the document.

2

17

195196197198

199

200201202

203204205206207208209

210211212213214

215216217218219

220221222

223224225226227228229

230231232233

234235236

Figure 1Common Alarms and Events Document Development Process

1.4 Document OrganizationSection 2 describes the operating environment of AMI systems including a high-level system architecture, security threats and objectives, and basic security event scenarios. Section 3 is the core of the document and describes categories of alarms and events that will support the development of common security objects. Section 4 describes a common security object development plan.

3

18

237

238239

240

241242243244245

19

2OPERATING ENVIRONMENTAMI systems enable utility back office networks to communicate with electric meters through Neighborhood Area Networks (NANs), backhaul networks, and AMI DeMilitarized Zone (DMZ) networks. Figure 2 depicts an abstract AMI network diagram. A typical interface within a NAN is a Radio Frequency Local Area Network (RFLAN) interface. These interfaces form a mesh network among electric meters and often use proprietary communication standards. A collector joins the RFLAN and also has an additional backhaul interface such as a cellular or Ethernet interface. This backhaul interface is used to communicate with an AMI DMZ network and the head-end system. The head-end system is connected to back office networks and can send down pricing signals as well as push firmware updates to electric meters and collectors.

The remainder of Section 2 discusses AMI security threats and objectives followed by security event scenarios.

Figure 2AMI Network Diagram (Courtesy Justin Searle, UtiliSec)

4

20

246

247

248249250251252253254255256

257258

259

260261

2.1 AMI Security Threats and ObjectivesThis section lists incidents that are both malicious and non-malicious events. Since the vast majority of cyber incidents on AMI systems are non-malicious, incident reports should be correlated to detect suspicious events and limit false positives.

Threats to AMI implementations include non-malicious actions and events caused by:

Operational mistakes

Employees who bypass security for convenience

Safety system failures

Equipment failures

Natural disasters

Threats to AMI implementations include malicious actions and events caused by:

Insider threats including disgruntled employees

Vandalism (random and malicious)

Loss of information privacy

Extortion

Cyber hacking

Viruses and worms

Theft (theft of device and theft of energy)

Terrorism

Security goals for AMI security event management systems include:

Detection of malicious actions

Assessment and characterization of events

Notification of appropriate system operators

Enabling appropriate responses to events

5

21

262

263264265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

22

3CATEGORIES OF ALARMS AND EVENTSThis section contains information on the common alarms and events to be implemented on the electric meter. Each system security requirement category is listed in bold typeface. For each system security requirement category, alarms, and events are grouped under sub-categories denoted in bold and italicized typeface. The system security requirement categories were selected from the categories listed in the AMI System Security Requirements document. The sub-categories were selected by evaluating the identified common alarms and events and grouping them under major components of the AMI.

Within this section, the following terminology is used to differentiate between “alarms” and “events”:

Alarms are considered security critical and should be reported immediately to the system operator or security event management system. Alarms and events may have direct impact on critical functions such as system control.

Events may or may not be security related but are useful as forensic evidence during the investigation of an AMI cyber-security incident. Correlated events may also indicate an attack in progress. Events are stored locally on the device and periodically sent to the event management system. For each device, events can happen at different logical layers within the system including network communication, operating system, or application.

This section contains tables describing the alarms, and events. Each table contains the following fields: name, description, type, and traceability. The name field is a short, unique way to identify the alarm, or event. The description field contains more information or a definition. The type field is populated with an A for an alarm and an E for an event. This traceability is documented in each table, or marked as not applicable when appropriate.

When possible, these alarms and events categories are traced to the requirements listed in AMI System Security Requirements and NISTIR 7628. The AMI System Security Requirements categories that are applicable to alarms and events consist of the following:

Authentication (FAT)

Integrity (FIN)

Accounting (FAC)

Anomaly Detection Services (FAS)

Cryptographic Services (FCS)

Notification and Signaling Services (FNS)

6

23

286

287

288289290291292293294

295296

297298299

300301302303304

305306307308309

310311312

313

314

315

316

317

318

The AMI System Security Requirements topics that are not applicable to alarms and events consist of the following:

Confidentiality and Privacy (FCP)

Availability (FAV)

Identification (FID)

Authorization (FAZ)

Non-Repudiation (FNR)

Boundary Services (FBS)

Resource Management Services (FRS)

Trust and Certificate Services (FTS)

Development Rigor (ADR)

Organizational Rigor (AOR)

Handling/Operating Rigor (AHR)

Accountability (AAY)

Access Control (AAC)

In cases where an alarm, or event traces to multiple items in AMI System Security Requirements, the following convention is used: ABC.1,2,..,N where ABC denotes the AMI System Security Requirements category code and 1-N denotes the specific requirement in the category. In cases where multiple categories are referenced, a semicolon (;) is used to separate the category (e.g., ABC.1,2; DEF.1.3).

The NISTIR 7628 categories that are applicable to alarm and event consist of the following:

Access Control (SG.AC)

Audit and Accountability (SG.AU)

Identification and Authentication (SG.IA)

Communication Protection (SG.SC)

Information Integrity (SG.SI)

In cases where an alarm, or event traces to multiple items in NISTIR 7628, the following convention is used: AB.CD-1,2,..,N where AB.CD denotes the category code and 1-N denotes the specific requirement in the category. In cases where multiple categories are referenced, a semicolon (;) is used to separate the category (e.g., AB.CD-1,2; EF.GH-1.3).

This section also includes material related to the Security Profile for AMI. If applicable, these relationships are indicated along with the section of that document referenced.

3.1 AuthenticationThis section describes alarms, and events related to authentication.

7

24

319320

321

322

323

324

325

326

327

328

329

330

331

332

333

334335336337338

339

340

341

342

343

344

345346347348

349350

351

352

25

3.1.1 C12.XXThe ANSI C12.18, C12.19, and C12.22 communication standards are implemented in the majority of AMI electric meters in North America. As it relates to security, the ANSI standards provide specifications for authentication controls on the meter. C12.XX alarms and events are included in Table 1.

These communication standards may be used to access AMI electric meters using field tools. Additional guidance regarding field tools is included in the Security Profile for AMI Section AMISP-2.10.1. Additional guidance regarding system connection is included in the Security Profile for AMI Section DHS-2.8.18.

C12.22 is a standard generally used in North America. In subsequent versions, this document will be generalized and include DLMS/COSEM alarms and events.

Table 1C12.XX Alarms, and Events

Event Type ID Name Description Type Traces To

TAB.1 C12.18 Successful Login This event occurs when a user successfully logs in.

E FAC.3; SG.AC-3,4

TAB.2 C12.18 Failed Login This event occurs when a user attempts to log in but is not successful.

E FAC.3; SG.AC-3,4,8

TAB.3 C12.19 User Password Modified

This event occurs when a user password is modified in the meter’s C12.19 tables.

E FAC.3; SG.AC-21

TAB.4 C12.19 User Permissions Modified

This event occurs when a user's permissions are modified in the meter’s C12.19 tables.

E FAC.3; SG.AC-3,4

TAB.5 C12.19 Least Privilege Alert This alert occurs when a user logs in with a lower privilege password and attempts to modify C12.19 tables that require a higher privilege.

A FAC.3; SG.AC 7

TAB.6 C12.22 Failed Authentication

This event occurs when a message authentication fails. This may occur as a result of a meter receiving a message with missing or invalid digital signatures for authentication.

E FAC.3; SG.AC-3,4,8; SG.SC-8,9,20

TAB.7 C12.22 Successful Login This event occurs when a user successfully logs in C12.18 over C12.22.

E FAC.3; SG.AC-3,4

TAB.8 C12.22 Failed Login This event occurs when a user attempts to log in

E FAC.3; SG.AC-3,4,8

8

26

353

354355356357

358359360361

362363

364365

C12.18 over C12.22 but is not successful.

3.2 IntegrityThe following section describes alarms and events related to integrity.

3.2.1 Event Log Storage and ManagementAlarms and status events related to event log storage and management are included in Table 2. The purpose of the following events is to document the operation and use of event storage.

Table 2Event Log Storage and Management Alarms and Events


LOG.1 Local Event Store Cleared This event occurs when the local event store is cleared.

E FAC.23

LOG.2 Local Event Log Overflowed

This event occurs when the local event log overflows. This may occur as a result of the meter receiving a high number of events.

A FAC.23; SG.AU-4,5

LOG.3 Event Log Configuration Change

This event occurs when the event log configuration changes. Types of configuration changes may include types or number of events logged.

E FAC.23; SG.CM-3,4,5,6

LOG.4 Event Log Disable This event occurs when the event logging functionality is disabled.

E FAC.23; SG.CM-6

9

27

366

367

368

369370

371372

28

3.2.2 Usage DataAlarms and status events related to the integrity of customer-specific usage data are included in Table 3.

Table 3Usage Data Alarms and Events


USG.1 Usage Data Cleared This event occurs when usage data is cleared from the device.

E FAC.23

USG.2 C12.19 Energy Usage Data Table Modified

This event occurs when the C12.19 usage data tables are modified on the device.

E FAC.23

USG.3 Usage Data Configuration Change

This event occurs when there is a usage data configuration change on the device. Types of usage data configurations may include consumption units and multipliers.

E FAC.23

3.2.3 Meter ConfigurationAlarms and status events related to the configuration of the AMI meter are included in Table 4.

Table 4Meter Configuration Alarms and Events


CFG.1 C12.19 Device Configuration Change

This event occurs when a device's configuration is changed in the meter’s C12.19 tables.

E SG.CM-3, 4, 5

3.3 ControlsThe section describes the alarms and events as it relates to the accountability of control functions on the meter.

3.3.1 Meter Disconnect SwitchThe meter disconnect switch enables the utility to send a signal to the AMI electric meter to disconnect a customer’s power if they are moving, have not paid their electric bills, or for safety purposes. Alarms and events related to triggering must be in place to detect malicious activity. Alarms and status events related to the meter disconnect switch are included in Table 5.

10

29

373

374375

376377

378

379

380381

382

383

384385

386

387388389390

Table 5Meter Disconnect Switch Alarms and Events


SWI.1 Remote Disconnect This alarm occurs when power is remotely disconnected. A high frequency of disconnects or many in a single neighborhood could indicate a problem.

A

SWI.2 Remote Reconnect This alarm occurs when power is remotely reconnected.

A

SWI.3 New Connect This event occurs when a new power connection is established.

A FAS.1,4

SWI.4 Local Disconnect This event occurs when power is locally disconnected. This event may be triggered through the meter’s C12.18 optical port interface.

A

SWI.5 Local Connect This event occurs when power is locally connected. This event may be triggered through the meter’s C12.18 optical port interface.

A

3.4 Anomaly Detection ServicesThis section describes the alarms and events as it relates to the anomaly detection functions on the meter.

3.4.1 MetrologyAlarms and status events related to detection of anomalies on the metrology board are included in Table 6.

Table 6Metrology Alarms and Status Events


MET.1 Abnormal Time Signal Readings

This event occurs when data communicated from the metrology board to the register board is received at an abnormal rate.

E FAS.1,4

11

30

391392

393

394395

396

397398

399400

31


MET.2 Metrology Failed Integrity Check

This event occurs when there is a failed integrity check in data communicated from the metrology chip to the register chip.

A FAS.1,4 SG.SC-8

3.4.2 Firmware/SoftwareAlarms and status events related to detections of anomalies in the firmware/software are included in Table 7. Guidance regarding firmware is included in the Security Profile for AMI Section DHS-2.10.

Table 7Software Alarms, and Events


FIR.1 Upgrade Initiation This event occurs when a firmware upgrade is initiated. The source of initiation should be included in the event.

E SG.AU-16

FIR.2 Software Failed Integrity Check

This event occurs when there is a failed integrity check concerning the firmware.

A FNS.1;FAS.1,4; ; SG.AU-2; SG.SI-7

FIR.3 Upgrade Complete This event occurs when the firmware is successfully upgraded.

E

FIR.4 Software Error This alarm occurs when there is a software error. Information regarding what type of error has occurred including buffer overflows, exceptions, and integrity check failures is included.

A FNS.1;FAS.1,4; SG.SI-7

3.5 Cryptographic ServicesThis section describes cryptographic services alarms and status events. Guidance regarding cryptographic services is included in the Security Profile for AMI Sections DHS-2.8.11, DHS-2.8.12, and DHS-2.8.15.

3.5.1 Key Management and CertificatesAlarms and events related to key management and certificates are included in Table 8. It is up to the vendor to provide sufficient information in addition to the defined alarms and events to distinguish the source of these key management and certificate alarms and events.

12

32

401

402403404

405406

407

408409410

411

412413414

Table 8Key Management Alarms and Status Events


CRY.1 Certificate Renewal This event occurs when the certificate renewal request is made before the certificate expires. The old key and certificate are retained until the new certificate is available.

E FCS.3,4; SG.IA-3; SG.SC-11,15

CRY.2 Key Generation This event occurs when a new key is generated on the device.

E FCS.3.,4; SG.IA-3; SG.SC-11

CRY.3 Key Update This event occurs when the key is updated.

E FCS.3,4; SG.IA-3; SG.SC-11,14

CRY.4 Key Revocation This event occurs when the key is revoked.

E FCS.3,4; SG.IA-3; SG.SC-11

3.6 Notification and Signaling ServicesThis section describes alarms and status events related to notification and signaling services.

3.6.1 Communications InterfacesAlarms and status events related to general communication interfaces are included in Table 9. Guidance regarding communications interfaces can be found in the Security Profile for AMI Section DHS-2.8.18. These interfaces could include the following:

Ethernet

Cellular

RF meshTable 9

Communication Interface Specific Events

13

33

415416

417

418

419

420421422

423

424

425

426427

34


COM.1 Communications Established This event occurs when a communications connection is established or re-established. This event may be used to detect signal interference or jamming attacks.

E FNS.2,3

COM.2 Communications Lost This event occurs when an existing communications connection is broken. This event may be used to detect signal interference or jamming attacks.

E FNS.2,3

COM.3 Communications Integrity Check Failure

This event occurs when an integrity check fails. This event may be used to detect signal interference or packet injection attacks.

E FNS.1,2,3;FAS.1,4; SG.SI-8

3.6.2 System Security Alarms and EventsAlarms and status events related to security are included in Table 10.

Table 10System Security Alarms and Events


SEC.1 C12.22 Replay Detected This alarm occurs when a C12.22 replay is detected.

A FNS.6

SEC.2 C12.22 Invalid Packet This alarm occurs when an invalid C12.22 packet is received.

A FNS.6; SG.SI-7; SG.SC-20

SEC.3 C12.22 Registration This event occurs when a C12.22 registration occurs.

E FNS.2,3;FAC.3; SG.IA-5

SEC.4 C12.22 Deregistration This event occurs when a C12.22 deregistration occurs.

E FNS.2,3;FAC.3; SG.IA-5

SEC.5 Time Synchronization Failed This event occurs when time synchronization fails.

E FNS.2,3

14

35

428

429

430431

432

3.6.3 Device Alarms and EventsAlarms and events related to the device are included in Table 11. The following alarms and events document changes in the physical status of the AMI components.

Table 11Physical and Device Alarms and Events


PHY.1 Inversion Tamper This alarm occurs when an inversion tamper is detected.

A FNS.1; FAS.2,3

PHY.2 Removal Tamper This alarm occurs when a removal tamper is detected.

A FNS.1; FAS.2,3

PHY.3 Reverse Rotation This alarm occurs when a reverse rotation tamper is detected.

A FNS.1; FAS.2,3

PHY.4 Battery Voltage Low This event occurs when the device's battery becomes low.

E

PHY.5 Device Power On This event is generated when power is restored.

E FNS.2,3

PHY.6 Device Reset This event is generated when the device has been reset via a hardware reset action.

E FNS.2,3

PHY.7 Device Outage This event is generated when a loss of power is detected. This event may be generated if a backup power source (e.g., battery or capacitive) is provided.

E FNS.2,3

PHY.8 Electromagnetic Attack This alarm occurs when an electromagnetic attack is detected. This may occur when an attempt is made to alter energy usage by placing a magnet on the meter.

A FNS.1; FAS.2,3

15

36

433

434435

436437

37

4COMMON SECURITY OBJECT DEVELOPMENT PLANThis section describes the general issues related to the formalization of the AMI alarms and events document and the path towards the development of an accepted definition of the structure, content, and semantics of AMI alarms and events, or common AMI security objects. Specifically, a common set of attributes must be ascribed to each and alarm and event, and a set of requirements for retention capabilities must be agreed upon.

4.1 AttributesAn alarm or event object should contain sufficient information for an event management system or system operator to make an informed decision about how to react to a security or system event. Every alarm and event should be authenticated at the source and linked to the identity of the device where it is generated. Basic attributes for all alarm and events are included in Table 12.

Table 12Attributes

Name Description Traces To

Event Type ID Event Type ID uniquely identifies the type of event that occurred.

FAC.7,8; SG.AU-3

Name Name is a word or short phrase that identifies the event.

SG.AU-3

Originator ID Originator ID identifies the source of the event. The Originator ID should contain information that allows the device type and equipment identity (e.g. serial number.)

FAC.8,9; SG.AU-3

Event Properties Event-specific properties which provide additional information about the security event. Examples include a disconnect event including a property that indicates if the disconnect was invoked locally or remotely. Another example would be a security failure event that indicates whether the failure was related to an optical interface or WAN.

FAC.8,20; SG.AU-3

16

38

438

439

440441442443444

445

446447448449450

451452

Name Description Traces To

Time Stamp Time stamp information should conform to the following:

Identifies the date and time of occurrence

Should use ISO 8601 time format with the ability to record up to millisecond precision

Time stamps should be adequately synchronized across systems

FAC.8,30; SG.AU-3,8

Event Quality Event quality information should conform to the following:

Helps determine if the event data has possibly been tampered with including information such as time not synchronized, physical tamper triggered, and device errors

FAC.13,21; SG.AU-9

Description Description includes a brief textual description of the event type

FAC.20; SG.AU-3

4.2 Retention CapabilitiesThis section describes a set of best practices for the retention of alarms and event information. The Security Profile for AMI provides guidance regarding backup and recovery in its section DHS-2.10.4. Additionally NISTIR 7628 includes related guidance in its section 3.9.

Best practices based on this guidance related to remote storage/backup capabilities include the following:

At minimum, an AMI electric meter should retain two weeks worth of data, but it should also be flexible and configurable to allow shorter or longer retention times based on regional and organizational regulations and policy.

At minimum, event logs should be transferred to a storage/backup system weekly before they are cleared locally on the meter.

Locally stored event log information should be readily correlatable to remotely stored event long information to allow for forensic investigation of meters without network connectivity. Log backups should be physically separated from the AMI system. At the head end, these log backups should be stored in information systems that are physically separate systems from the AMI system and in accordance with the organizations policies for enterprise data retention.

Log backups should be stored in a secure environment with controlled access and provisions for disaster recovery.

Regarding the format of audit records and storage, in order to encourage interoperability between event management systems and AMI vendors, logs should conform to an open standard such as the IETF SYSLOG protocol or HTTP with XML delineated data.

17

39

453

454455456

457458

459460461

462463

464465466467468469

470471

472473474

40

Data should be stored in an aggregated event management system, either distributed or centralized. The data should be accessible independently from the AMI system in order to support offline analysis.

4.3 Development RoadmapThe following is a proposed roadmap of the steps that must be taken to achieve the goal of the development of common AMI security objects.

1) Formalize AMI Common Alarms and Events Document

a. Engage key stakeholders in a public process to review the requirements, identify gaps, and formalize the document.

b. An existing industry working group such as OpenSG would be preferred for this process.

2) Develop Stakeholder and Vendor Consensus

a. Solicit key stakeholders, such as asset owners, AMI vendors, and security information and event management vendors to form a working group tasked with the development of common AMI security objects.

b. Once a final set of requirements has been developed, the document shall be presented to a working group body for review and formalization.

c. The development of common AMI security objects will be beneficial to all parties as they allow for more resources to be applied towards detection of threats, as opposed to integration of systems.

3) Facilitate the Development of AMI rule sets for SIEM systems

a. Engage security application vendors in the development of AMI rule sets for existing SIEM systems.

b. Provide expertise and guidance regarding standards and interoperability specific to AMI rule sets for SIEM systems.

4) Develop Conformity Testing and Certification Framework

a. A conformity testing and certification testing framework shall be developed to ensure interoperability between SIEM systems that have AMI-specific functionality.

b. The framework shall be verified in a laboratory environment.

18

41

475476477

478

479480

481

482483

484485

486

487488489

490491

492493494

495

496497

498499

500

501502503

504

505

5CONCLUSION AND NEXT STEPSThis document has identified a set of common AMI alarms and events. Based on this information, a set of common AMI security objects will be defined and documented. The development roadmap in Section 4 lays forth the path towards formalization of this document and development of the common AMI security objects.

The next steps in this process will be to develop a schedule to complete the roadmap identified in Section 4. This schedule will culminate with EPRI project P183.009, Standardized Security Objects for AMI. P183.009 intends to develop standard security objects for AMI systems. Standardized Security Objects for AMI is expected to build on AMI incident response work that was performed in the PDU Cyber Security and Privacy Initiative. In the Initiative, EPRI coordinated with major AMI vendors to identify specific alarms and events for which standard security objects should be developed. Standardized Security Objects for AMI will be developed based on a consensus among the vendors for how each of the alarms and events should be represented. This accepted definition of the structure, content, and semantics of AMI alarms and events will enable a new generation of AMI SIEM systems. Third-party security application vendors may also be included to accelerate the development of AMI SIEM systems. Finally, the interoperability of the AMI vendors' implementations is intended to be verified in a laboratory environment.

19

42

506

507

508509510511

512513514515516517518519520521522523524

43

6APPENDIX: REFERENCES, GLOSSARIES, AND INDEXESThis section will contain a listing of the document's supporting references, a glossary, and an index.

6.1 ReferencesThis section will contain a listing of the document's supporting references.

IEC/TS 62351-7 TS Ed.1 (2008-04): Power systems management and associated information exchange -Data and communication security - Part 7: Network and system management (NSM) data object models

6.2 AcronymsAMI - Advanced Metering InfrastructureAMI-SEC - AMI SecurityANSI - American National Standard InstituteASAP-SG - Advanced Security Acceleration Project for the Smart GridCBKE - Certificate Based Key ExchangeCSV - Comma Separated ValueC12.18 standard - ANSI standard for type 2 optical portC12.22 standard - ANSI specification for interfacing to data communication networksCSWG - Cyber Security Working GroupDPA - Differential Power AnalysisDMZ - DeMilitarized ZoneHAN - Home Area NetworkHTTP - HyperText Transfer ProtocolIDS - Intrusion Detection SystemIEC - International Electrotechnical CommissionIETF - Internet Engineering Task ForceIT - Information TechnologyMAC - Media Access ControlMDMS - Meter Data Management SystemMIB - Management Information BasesNAN - Neighborhood Area NetworksNESCOR - National Electric Sector Cybersecurity Organization ResourcesNIST - National Institute of Standards and TechnologyNIST CSWG - NIST Cyber Security Working GroupNISTIR - National Institute of Standards and Technology Interagency Report- Protocol Data Unit (Also – power delivery and utilization)RFLAN - RF Local Area NetworkSGIP-CSWG - Smart Grid Interoperability Panel Cyber Security Working Group

20

44

525

526

527

528529

530

531

532533534

535

536537538539540541542543544545546547548549550551552553554555556557558559560561562563

SIEM - Security Information and Event ManagementSOAP - Simple Object Access ProtocolSPA - Simple Power AnalysisSYSLOG - IETF standard for computer data loggingWG - Working GroupXML - eXtensible Markup LanguageZigBee - HAN communication protocol

21

45

564565566567568569570571

46

7APPENDIX: MALICIOUS AMI SECURITY EVENT SCENARIOSMalicious AMI security events consist of compromise attempts, a potentially successful malicious action, and symptoms of the attack. This section describes two possible security event scenarios as background: 1) a simple security event that can be detected with a single category of alarms and 2) a more complex event that requires correlating multiple security alarms.

Each scenario consists of a description of the attack, possible alarms triggered including an Event Type ID that can be used to refer to additional information in Section 3, and actions that the utility can take to mitigate the attack.

7.1 Scenario 1: Simple Physical Meter Attack7.1.1 AttackIn this scenario, an attacker attaches an optical probe and attempts to authenticate to the deployed meter using the ANSI C12.18 protocol. Once authenticated, the attacker attempts to obtain other credentials such as keys and passwords stored within the respective C12.19 tables in order to gain access to other systems. Figure 3 depicts this attack.

Figure 3Simple Physical Meter Attacks

7.1.1.1 Possible Alarms Triggered

The following alarms and event are triggered:

C12.18 Failed Login (TAB.2) – Triggered repeatedly as the attacker fails to gain access.

22

47

572

573

574

575576577578

579580581

582

583

584585586587

588589590

591

592

593

C12.18 Successful Login (TAB.1) – Triggered once when the attacker successfully gains access.

7.1.1.2 Utility Actions

After alarms are sent to the utility, a decision will be made to respond and send personnel to the location. These personnel may or may not be trained to assess whether other malicious attacks have occurred so if the equipment was not destroyed, it could be replaced by a new meter.

7.2 Scenario 2: Complex Cyber-Physical Meter Attack7.2.1 Attack Stage 1: Physical Removal of MeterAttacker gains physical access to the meter and removes it from the meter enclosure. Figure 4 depicts this attack stage.

Figure 4Complex Attack Stage 1



Removal Tamper (PHY.2) – Triggered once the meter is removed from its enclosure

Device Outage (PHY.7) – Triggered once the meter is disconnected from its power source

7.2.1.2 Utility Action

The alarms and events are received by the utility as the meter is being removed. A decision could be made to respond and send personnel to the location

7.2.2 Attack Stage 2: Physical Attack of Meter HardwareIn the second stage, the attacker attaches hardware and tools to device to obtain security relevant information. Figure 5 depicts this attack stage.

23

48

594595

596

597598599

600

601

602603

604605606

607

608

609

610

611

612613

614

615616

49




Software Error (FIR.4) – Triggered as software errors occur while the attacker attempts to utilize various exploits.

Device Reset (PHY.6) – Triggered as the attacker resets the device while trying to read out the memory.


If the meter is reconnected to the system, the alarm will be received by the utility. A decision could be made to respond and send personnel to the location.

7.2.3 Attack Stage 3: Usage of Stolen CredentialsAs a result of reading out the memory the attacker has gained access to the C12.18 login credentials. The attacker has figured out a way to communicate with the meter over C12.22 and attempts to use the stolen credentials to log into the meter. After several unsuccessful attempts, the attacker logs into the meter and issues a remote disconnect command. Figure 6 depicts this attack stage.

24

50

617618619

620

621

622623

624625

626

627628

629

630631632633634




C12.22 Failed Login (TAB.8) – Triggered as the attacker makes unsuccessful login attempts.

C12.22 Successful Login (TAB.7) – Triggered as the attacker successfully logs in.

Remote Disconnect (SWI.2) – Triggered as the attacker issues a remote disconnect command.


The utility receives multiple C12.22 failed logins, a successful login, and a remote disconnect alarm. A decision could be made to respond and send personnel to the location.

25

51

635636637

638

639

640

641

642643

644

645

646647

52

Intrusion Detection System for Advanced Metering Infrastructure

Product ID Number

0

648

649

650

651

652

653

53

Intrusion Detection System for Advanced Metering Infrastructure

Product ID Number

Draft Release – August 10, 2012


654

655

656

657

658

659

660

661

662

663

54






University of Illinois at Urbana-Champaign





55

664

665666667668

669670671672673674

675676677678679

680681

682683684685686687688

689690691692693

694

695696

697698

699

56


University of Illinois at Urbana-Champaign1308 W. Main St.Urbana, Illinois 61801

Principal Investigator:Robin Berthier

This report describes research sponsored by the Electric Power Research Institute (EPRI).



700

701702

703704705

706707

708

5758

59

60

ABSTRACTThe deployment of Advanced Metering Infrastructures (AMI) significantly increases the attack surface that utilities have to protect. As a result, there is a critical need for efficient monitoring solutions to supplement protective measures and keep the infrastructure secure. This document investigates current industrial and academic efforts to address the challenge of detecting security events across the range of AMI networks and devices. The goal of this study is to help utilities and vendors to understand intrusion detection requirements, gaps in existing approaches, and research problems that need to be solved to build and deploy a scalable and comprehensive security monitoring solution.

v

61

709

710711712713714715716717

62

CONTENTS1 INTRODUCTION.....................................................................................................................1

1.1 Purpose and Scope........................................................................................................11.2 Document Organization..................................................................................................1

2 MONITORING REQUIREMENTS AND CURRENT APPROACHES......................................22.1 AMI Security Threats and Monitoring Requirements......................................................22.2 Major Security Concerns.................................................................................................32.3 Industry Solutions...........................................................................................................32.4 Academic Solutions........................................................................................................5

3 GUIDELINES FOR A SCALABLE AND COMPREHENSIVE IDS FOR AMI.........................83.1 Characteristics of an IDS Architecture for AMI...............................................................83.2 Case Study.....................................................................................................................9

3.2.1 Intrusion Detection Operations Required...............................................................93.2.2 Monitoring Architecture Components, Topology, and Communications..............103.2.3 Alert Correlation and Aggregation........................................................................12

4 CONCLUSION AND NEXT STEPS.......................................................................................135 APPENDIX: REFERENCES, GLOSSARIES, AND INDEXES.............................................14

5.1 References....................................................................................................................145.2 Acronyms......................................................................................................................15

vi

63

718

719720721

722723724725726

727728729730731732

733

734735736

737

LIST OF FIGURESFigure 1: Percentages of IDS vendors for different technologies and environments. Source: publicly available information from top 15 smart grid security solution vendors............................4Figure 2: Characteristics of a scalable and comprehensive intrusion detection system for AMI...9Figure 3: AMI Network Diagram Instrumented with IDS Components (Courtesy Justin Searle, UtiliSec).......................................................................................................................................11

vii

64

738

739740741742743744

745

65

LIST OF TABLESTable 1: Overview of research publications related to IDS for AMI or SCADA environments.......5Table 2: Monitoring Operations and Sensor Placement Based on Attack Consequences.........10

ix

66

746

747748749

750

67

1INTRODUCTION

1.1 Purpose and ScopeThis Intrusion Detection Systems (IDSes) for Advanced Metering Infrastructure (AMI) document is a product of the EPRI AMI Incident Response Project. The document is intended to give AMI vendors and asset owners a clear understanding of the unique monitoring requirements of AMI and to identify key research challenges related to intrusion detection technology and large-scale deployment.

The effective design and deployment of IDSes in a utility’s AMI environment have several characteristics that differentiate them from design and deployment in traditional information technology (IT) environments. For example, simply deploying a perimeter IDS may not provide the coverage necessary for an AMI system. Since there tend to be mesh networks in addition to IP-based backhaul networks, positioning an IDS at the AMI head-end system could miss malicious activity in the mesh network. In addition, there can be scalability issues, as some utilities deploy millions of meters in their service territories.

The scope of this document includes monitoring requirements for the core components of an AMI (i.e., collection engine, meter data management system, data collection unit, and meters) and does not cover the home area network (HAN) or third-party communication equipment.

1.2 Document OrganizationSection 2 reports on requirements for monitoring the security of AMI and on existing approaches from both industry and academia to address those requirements. This review leads to a gap analysis presented in Section 3, where key research challenges and guidelines for deploying a scalable IDS for AMI are identified.

751

752

753

754755756757758

759760761762763764765

766767768

769

770771772773

2MONITORING REQUIREMENTS AND CURRENT APPROACHES2.1 AMI Security Threats and Monitoring RequirementsThe deployment of an AMI represents a significant increase in the attack surface that utilities have to protect. The addition of a communication infrastructure and the processing capabilities of AMI devices coupled with the physical accessibility of smart meters and even access points enable new ways to penetrate the system and could attract a wide array of threats. Among the attack motivations that are specific to AMI, we consider:

Energy fraud

Service disruption for the purposes of extortion (e.g., through a denial of service), vandalism, hacktivism, or terrorism (e.g., power disruption through remote disconnects)

Theft of sensitive information

Abuse of communication infrastructure (e.g., by creating a botnet)

Malicious activities that achieve those goals could have a heavy financial impact on utilities and would likely result in major losses of customer trust and technology adoption. As a result, it is critical that utilities have a way to perform timely detection and identification of malicious actions and incidents so that local issues can be mitigated before they escalate. This objective requires the implementation of an efficient monitoring solution.

The challenges to address when designing a monitoring solution for a large and complex system include:

What information should be collected?

Where should sensors be deployed, and how can visibility over the information required for detection be gained?

Which detection technologies would be best suited to triggering alarms when malicious activity occurs?

How should appropriate system operators be notified?

Should a separate communication channel be used to exchange intrusion detection information and configuration?

Which data aggregation and correlation techniques should be deployed to optimally differentiate malicious events from legitimate ones?

The last question is crucial, because the likelihood of legitimate events that could trigger intrusion detection alerts is high. For instance, security alarms could be triggered because of operational mistakes, misconfigurations, system failures, or disruptive events such as natural disasters. The misidentification of legitimate failures as malicious actions would generate false positives and lessen the efficacy of the monitoring solution.

1

68

774

775

776

777

778779780781782

783

784785

786

787

788789790791792

793794

795

796797

798799

800

801802

803804

805806807808809

2.2 Major Security ConcernsAs part of the initiative to develop common alerts and alarms for AMI, and to understand the needs for AMI cyber security incident detection and response, a questionnaire was developed and sent to utility partners. Respondents represented a diversity of environments (urban, suburban, and rural) and deployment phases (pilot planned, started, and completed). The top security concerns expressed were loss of controllability over AMI devices, followed by loss of observability due to a lack of data integrity. Cyber threats specific to AMIs included meter compromise and massive remote disconnects. Indeed, meters, along with pole-top collectors, are the components that are most vulnerable to cyber intrusion by an external entity, while the head-end system and vendor access are seen as the most vulnerable to insider attacks.

At a lower level, utilities expressed concerns with respect to the following security events: Unauthorized massive remote disconnect Device tampering: malware and malicious code injection (e.g., through buffer overflow

attack attacks), rogue device attachment, meter tampering, access to firmware password, and zero-day attacks against AMI devices

Encryption issues: access to decryption keys or discovery of flaws in encryption Denial of service against routers or cell relays Unauthorized modifications to system configurations and physical components

This list offers an initial guide to understanding the need for a comprehensive monitoring solution and identifying where intrusion detection sensors should be deployed. For example, the importance of threats targeting devices in the field indicates that the integrity and health of AMI devices should be closely monitored. However, instrumenting and monitoring every device may be too expensive, and, as explained in the following sections, current security solutions indeed do not cover this requirement.

2.3 Industry SolutionsUtilities are investing in monitoring solutions to complement the anti-tampering alarms and event-logging capabilities already offered by smart meters. As shown on Figure 7, a survey of the 15 leading security vendors that offer AMI monitoring solutions showed that products mostly fall into two categories:

1. Centralized network-based intrusion detection sensors, and2. Centralized security information and event managers (SIEMs).

810

811812813814815816817818819820821822823824825826827828829830831832833834835

836

837838839840841842843

Host-based sensor for embedded device

Network-based monitoring sensor

Centralized SIEM in utility network

Field monitoring solution for AMI

Field monitoring solution for SCADA

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Figure 7: Percentages of IDS vendors for different technologies and environments. Source: publicly available information from top 15 smart grid security solution vendors.

Network-based IDSes are sometimes coupled with a firewall to gain intrusion prevention capabilities, and sit in the head-end behind decryption servers to have access to clear traffic. Either they perform packet header analysis only, or they also include application-level dissectors to analyze payloads. SIEMs are also installed in the utility network and receive logs from security appliances and devices using Syslog and a variety of information sources. They offer a central database to ease event aggregation, correlation, and visualization across components and over time.

Those products offer a cost-effective solution to monitoring events and communication traffic from a large volume of AMI devices. Most of them were designed for SCADA monitoring, but an increasing number now integrate AMI protocol analysis capabilities and data-mining approaches to process AMI events. While those products are important for monitoring the infrastructure, the cost advantage of deploying only a centralized solution has to be weighed against the limitation of not having visibility over events that occur at the edge of the network. In particular, neighborhood area networks (NANs) are usually deployed with a wireless mesh communication infrastructure, in which a significant portion of the traffic occurs among network nodes and is invisible to monitoring devices at the head-end. As a result, threats such as unauthorized remote disconnects originating from the field cannot be detected by current centralized solutions. Indeed, utilities have expressed the need for security solutions that could provide situational awareness over all parts of the infrastructure. Another security monitoring gap emphasized by utilities has been the lack of host-based intrusion detection sensors embedded on AMI devices to permit remote checking of firmware integrity and to identify compromised devices. In addition, utilities expressed strong interest in large-scale patch management solutions for embedded devices deployed in the field.

The responses to the questionnaire on AMI Incident Response Guidelines and Best Practices indicate that the main reasons for the industry’s current push for centralized monitoring solutions and the lack of distributed IDSes in the field have been 1) the need for high cost efficiency, 2) the lack of maturity of AMI security (e.g., how to assess the likelihood and criticality of a smart meter compromise), and 3) the difficulty of integrating proprietary communication protocols (e.g., most mesh network communication technologies at the lower layers are proprietary).

3

69

844845846

847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878

2.4 Academic SolutionsResearch from academic institutions and national labs can be organized into three categories: 1) efforts to understand the threat environment, 2) efforts to develop security monitoring architectures, and 3) design of network- or host-based intrusion detection sensors. Those research efforts are summarized in Table 13 and detailed below.

Table 13: Overview of research publications related to IDS for AMI or SCADA environments

Publications Threat analysis

Host IDS Network IDS

AMI SCADA

[1] Energy theft in the AMI (McLaughlin, 2010) ✓ ✓

[2] Multi-vendor penetration testing in the AMI (McLaughlin, 2010) ✓ ✓

[3] Cyber security issues for AMI (Cleveland, 2008) ✓ ✓

[4] Intrusion detection for AMI: requirements and architectural directions (Berthier, 2010)

✓ ✓ ✓ ✓

[5] Cumulative attestation kernels for embedded systems (LeMay, 2009) ✓ ✓

[6] An IDS for wireless PCS (Roosta, 2008) ✓ ✓

[8] Intrusion monitoring in PCS (Valdes, 2009) ✓ ✓ ✓

[9] Intrusion detection in SCADA networks (Barbosa, 2010) ✓ ✓

[10] Sophia proof of concept report (Rueff, 2010) ✓ ✓

[11] Distributed IDS in a multilayer network architecture of smart grids (Zhang, 2011)

✓ ✓ ✓

[12] Specification-based IDS for HAN (Jokar, 2011) ✓ ✓

879

880881882883884

885

[13] Specification-based IDS for AMI (Berthier, 2011) ✓ ✓

In the first category, [3] reviewed the security requirements for AMIs and the related threats, emphasizing that encryption and authentication alone will not be sufficient security protections, and that monitoring solutions are a critical complement. [1] provided a detailed security analysis of the issue of energy theft. The authors explained that AMI would significantly increase the risk of energy theft because of the interconnected nature of the infrastructure and the large-scale deployment of identical devices, leading to an amplification of effort, a division of labor, and an extended attack surface. In [2], the same authors introduce a penetration testing method to evaluate AMI components, revealing vulnerabilities such as the sending of unencrypted passwords over optical ports, the possibility of replaying authentications, and the derivation of encryption keys from meter passwords.

In the category of work on security monitoring architectures, [4] outlines the requirements for a comprehensive intrusion detection system for AMI, based on an analysis of the threat model and the information required for detection. In particular, the authors explained that specification-based intrusion detection systems that enable the deployment of a white-listed network would offer a strong security monitoring solution in an AMI environment where communications are tightly controlled and deterministic. This assumption of well-behaved network traffic is characterized in [9], in which the authors explained how the fixed number of network devices, the limited number of protocols, and the regular communication patterns found in the SCADA environment would enable precise network traffic models that can be leveraged for intrusion detection. That notion was used to build a tool called Sophia [10] that captures network traffic in industrial control systems to build a baseline model, and then triggers alerts when deviations are detected. [8] used a combination of specifications, change detection, and statistical anomalies to monitor process control systems protocols such as Modbus and DNP3. Once again, taking advantage of the regularity of network communication patterns enables the hybrid approach to detect both known and unknown attacks. [11] described a distributed intrusion detection system for both AMI and SCADA systems that relies on anomaly-based sensors deployed in HAN, NAN, the WAN, and SCADA environments. The sensors collect security-relevant information from the communication flows, and two machine-learning algorithms, including a support vector machine and an artificial immunity approach called clonal selection, process the data to identify malicious behavior. Those algorithms offer high detection accuracies if they are correctly and sufficiently trained.

Finally, in the category of work on host-based intrusion detection sensors, [5] introduces an architecture called the cumulative attestation kernel that addresses the issue of securely auditing firmware updates in embedded systems such as smart meters. The system is designed to be cost-, power-, computation-, and memory-efficient. A prototype is implemented to demonstrate the feasibility of the solution as well as to formally prove that it meets remote attestation requirements. With respect to network-based intrusion detection systems, [6] proposes a model-based sensor working on top of the WirelessHART protocol to monitor and protect wireless process control systems. The hybrid architecture consists of a central component that collects information periodically from distributed field sensors. A set of eight detection rules working on

5

70

886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928

the physical, data-link, and routing layers covers threats including signal jamming, node compromise, and packet modification. [12] presents and evaluates a specification-based intrusion detection sensor for HAN designed to monitor the physical and MAC layers of the ZigBee protocol. [13] also took advantage of a specification-based approach to monitor the ANSI C12.22 protocols through dedicated sensors deployed in the NAN. That solution was unique in using formal methods to prove that specification-based checkers offer sufficient coverage with respect to an AMI security policy.

929930931932933934935936937938939940941942

3GUIDELINES FOR A SCALABLE AND COMPREHENSIVE IDS FOR AMI3.1 Characteristics of an IDS Architecture for AMIThe study of the threat model, the needs expressed by utilities, the current solutions from security providers, and the latest research efforts on AMI monitoring provide a set of key insights into the characteristics of a comprehensive IDS for AMI. Those characteristics are summarized in Figure 8 and described below.

1. Monitoring of AMI communications at the head-end is necessary but not sufficient. Important threats that occur at the edge of the network mean that it is also necessary to instrument field devices or to deploy sensors in the field.

2. Monitoring of embedded operating systems in devices deployed in the field with host-based intrusion detection systems is critical. It empowers security operators to validate security alerts by checking whether the integrity of devices has been altered. This capability should be combined with an efficient patch distribution and management mechanism.

3. Network-based intrusion detection systems should leverage the deterministic nature of energy system communications through the implementation of a white-list approach in order to gain higher detection accuracy, to handle unknown attacks, and to work without the need for frequent updates.

4. IDS developers should embrace formal verification tools to validate the design of checkers for both host- and network-based intrusion detection systems. Those tools have been successfully used to check hardware design of critical systems, and they can offer strong mathematical guarantees to ensure that the stringent security requirements of AMI are met.

5. The deployment of IDS sensors in the field requires strong protection mechanisms and separate communication channels to prevent the IDS from becoming compromised. In addition, trust-building schemes such as majority voting should be implemented to ensure that attackers cannot easily forge alerts.

6. The monitoring architecture should scale to AMIs made of millions of devices. This high scalability requirement means that distributed detection technologies should be favored, in addition to smart data aggregation schemes that would enable operators to gain situational awareness without being overwhelmed by the number of alerts.

7. Finally, any security solution deployed in the smart grid environment has to be highly practical by reinforcing security layers without affecting the core mission of delivering energy. This requirement also applies to the monitoring architecture, which means that autonomous sensors and self-learning algorithms should be leveraged.

7

71

943

944

945

946

947948949950

951952953

954955956957958

959960961962

963964965966967

968969970971

972973974975

976977978979

Figure 8: Characteristics of a scalable and comprehensive intrusion detection system for AMI

3.2 Case StudyBased on the characteristics and requirements outlined in the previous subsection, we now illustrate the design of a scalable and comprehensive IDS architecture for AMI through an example. We assume a traditional AMI architecture made of two types of network: a back-haul connecting the utility to a set of collectors deployed in the field, and neighborhood area networks to connect meters to collectors.

3.2.1 Intrusion Detection Operations RequiredIn order to define detection technologies and sensor placement, the first step is to translate the threat model into attack consequences. Those consequences are key to understanding the information required by an intrusion detection system for the successful identification of attacks.

In Table 1, which has been updated from one provided in [4], monitoring operations are defined based on a generic but comprehensive list of attack consequences, and organized according to three detection technologies:

Stateful specification-based monitoring: to track the behavior of nodes in the network over time and to compare operations to protocol specifications and security policy in order to flag deviations (e.g., by validating the sequence of C12.22 requests and responses and by monitoring the frequency of critical operations such as remote disconnects).

Stateless specification-based monitoring: to verify a security property (e.g., the integrity of firmware, or the correct format of a C12.22 payload) without having to keep a state over time.

Anomaly-based monitoring: again, to verify security properties, but with respect to statistical metrics (e.g., network bandwidth) rather than detailed system specifications (e.g., by monitoring the signal power level and packet losses in wireless mesh networks).

The locations of sensors are defined by the information accessible at each location (head-end, collectors, or meters) and the processing capabilities available and required by the monitoring operations. Typically, stateful monitoring operations require more computations than stateless operations. Finally, in the case of network-based monitoring, the rightmost column provides indications of which protocol layers of the OSI model have to be monitored.

980981

982

983984985986987

988

989990991

992993994

995996997998999

100010011002

100310041005

10061007100810091010

Table 14: Monitoring Operations and Sensor Placement Based on Attack Consequences

Attack Consequences Detection Goal and Operation Sensor Locations

Protocol Layers (OSI

Model)Stateful Specification-based Monitoring

Integrity of configuration and routing protocols

Checking of configuration and routing operations against security policy and network configuration

Collectors 3-4

Illegitimate network operations

Stateful checking of protocol operations against security policy and application configurations

Collectors and/or head-end 5-7

Stateless Specification-based MonitoringInconsistent traffic origin or destination

Checking of packet header against security policy and network configuration


Integrity of communication traffic

Checking of packet payload against protocol specifications


Illegitimate use of credentials Checking of system logs against security policy Head-end 5-7

Integrity of node software or hardware

Operating system, application, and file integrity checking

Meters, collectors, and head-end Host

Unresponsive nodes Checking of protocol operations against security policy and application configuration Collectors 2-7

Anomaly-based MonitoringHigh bandwidth usage

Traffic monitoring against normal statistical profiles

Meters and/or collectors 3-4

High signal power level

Checking of wireless signal against normal statistical profiles

Meters and/or collectors 1-2

3.2.2 Monitoring Architecture Components, Topology, and CommunicationsFollowing guidance on intrusion detection systems from [14], a monitoring architecture for AMI can be decomposed into the following components:

Sensors: software or hardware components to capture and analyze network or system activity. In the case of an AMI, sensors should be deployed at the head-end, collectors and meters. Head-end sensors would process a large volume of traffic, while sensors in meters should have minimum computing requirements.

Management server: information generated by sensors should be sent to one or several management servers. The roles of the management server are 1) to store events in a database, and 2) to run a correlation and aggregation process to detect intrusions that could not be identified locally.

Database server: repository for event information recorded by sensors and management servers. The combination of the management server and the database server is often called a Security Information and Event Management (SIEM). A SIEM can log security events from other sensors than the ones deployed in the AMI.

9

72

1011

1012

1013

10141015

1016101710181019

1020102110221023

1024102510261027

Console: interface that security administrators can use 1) to configure the intrusion detection systems, 2) to monitor the security state of the AMI, 3) to visualize and explore alerts, and 4) to conduct forensics activity.

Figure 9 presents the topology of the monitoring architecture with the locations of the various components in the AMI.

Figure 9: AMI Network Diagram Instrumented with IDS Components (Courtesy Justin Searle, UtiliSec)

Communications among IDS components should be isolated from metering traffic. At the head-end and in the backhaul, that can be achieved through an encrypted VLAN. In NANs, depending on the communication medium, IDS management traffic and alerts can be carried either on a separate protocol (e.g., XML or JSON over SSL) or through the AMI communication traffic (e.g., ANSI C12.22) but with separate encryption keys.

To reach high scalability, IDS sensors should preprocess collected activities and be as autonomous as possible in order to send only the most relevant information to the management

IDS console

Embedded sensor on meter

IDS sensors on collectorsdataba

mgmt Host-based IDS at head-end

Network-based IDS at head-end

SIEM

102810291030

10311032

103310341035

10361037103810391040104110421043

server. The management server should use machine-learning algorithms to correlate and aggregate events over time with the goals of 1) translating raw sensor data into actionable information, and 2) reducing false positive rates. Correlation and aggregation techniques are described in the next subsection. Additionally, for large-scale AMIs, the management server and sensors at the head-end can be deployed in a two-tier, load-balancing architecture, such that a first set of appliances preprocess (e.g., to decrypt payloads) and route monitoring traffic to the correct server for final processing and storage.

Finally, two important mechanisms enable the monitoring architecture to become more resilient: 1) reduction of the trust of individual sensors by requiring majority voting or event correlation across multiple sensors before triggering of alerts, and 2) removal of single points of failure through deployment of multiple management servers and/or distribution of IDS information among sensors (e.g., using distributed hash tables).

3.2.3 Alert Correlation and AggregationThe significant size of AMI requires deploying highly efficient security event managers in order to process large volume of alerts while providing timely information about critical events and keeping a low volume of false positives. Alert processing operations [7] are organized according to the following steps:

1. Pre-processing: a. Normalization and storage of alerts into a standard format (e.g., IDMEF).b. Organization of normalized alerts into a relational database. Tables should be

created for AMI components (e.g., meters, collectors, routers, etc.) and events (e.g., C12.22 failed authentication, remote disconnect, etc.).

2. Aggregation: a. Computation of probabilistic similarity measures among alerts (e.g., across space,

such as several meters reporting high numbers of packet losses in the same NAN, and across time, such as collectors reporting scanning attempts over a similar period).

b. Reduction of the volume of alerts through clustering and merging following attribute analysis and similarity measures (e.g., alerts targeting the same ApTitle), or through filtering following rules learned over time (e.g., discarding C12.18 authentication alerts if related to approved maintenance operations).

3. Correlation: a. Using predefined attack scenarios, specified by experts or learned over time (e.g.,

an energy theft attempt would likely combine anti-tampering alerts with outage notifications). This correlation approach is only effective for known attacks.

b. To complement the previous approach and handle unknown attacks, correlation of alerts can be made by linking attack steps over prerequisites and consequences of attacks (e.g., integrity violation of a meter system following a network buffer overflow exploit targeting the same meter).

c. Correlation through multiple information sources, by combining knowledge about policies (e.g., maximum frequency for remote disconnect operations), maintenance operations (e.g., configuring a meter with a field device), and alerts.

11

73

1044104510461047104810491050105110521053105410551056

1057

1058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088

4CONCLUSION AND NEXT STEPSThis document has identified a set of characteristics of a scalable and comprehensive monitoring architecture for AMI, based on the review of AMI threats, utility needs, security vendor solutions, and the research literature. Those characteristics were illustrated through a case study that presented IDS components along with a topology and a discussion about IDS communication architecture and alert correlation and aggregation techniques.

The next steps to improving the technology will be to work with vendors, utilities, and third parties to ensure the interoperability of IDS components for AMI through the identification of standard interfaces and standard communication protocols.

1089

1090

10911092109310941095

109610971098

5APPENDIX: REFERENCES, GLOSSARIES, AND INDEXES5.1 References[1] S. McLaughlin, D. Podkuiko, and P. McDaniel, “Energy theft in the advanced metering infrastructure,” in Proceedings of the Critical Information Infrastructures Security, pp. 176–187, 2010.

[2] S. McLaughlin, D. Podkuiko, S. Miadzvezhanka, A. Delozier, and P. McDaniel, “Multi-vendor penetration testing in the advanced metering infrastructure,” in Proceedings of the 26th Annual Computer Security Applications Conference, ACM, 2010, pp. 107–116.

[3] F. Cleveland, “Cyber security issues for advanced metering infrastructure (AMI),” in Proceedings of the Power and Energy Society General Meeting: Conversion and Delivery of Electrical Energy in the 21st Century, IEEE, 2008, pp. 1–5.

[4] R. Berthier, W. Sanders, and H. Khurana, “Intrusion detection for advanced metering infrastructures: Requirements and architectural directions,” in Proceedings of the First IEEE International Conference on Smart Grid Communications (SmartGridComm), IEEE, 2010, pp. 350–355.

[5] M. LeMay and C. Gunter, “Cumulative attestation kernels for embedded systems,” Proceedings of Computer Security–ESORICS 2009, pp. 655–670, 2009.

[6] T. Roosta, D. Nilsson, U. Lindqvist, and A. Valdes, “An intrusion detection system for wireless process control systems,” in Proceedings of the 5th IEEE International Conference on Mobile Ad Hoc and Sensor Systems (MASS 2008), IEEE, 2008, pp. 866–872.

[7] U. Zurutuza and R. Uribeetxeberria, “Intrusion detection alarm correlation: a survey,” in: Proceedings of the IADAT International Conference on Telecommunications and Computer Networks (TCN’04), Donostia, Spain, December 2004.

[8] A. Valdes and S. Cheung, “Intrusion monitoring in process control systems,” in Proceedings of the 42nd Hawaii International Conference on System Sciences, pp. 1–7, 2009

[9] R. Barbosa and A. Pras, “Intrusion detection in SCADA networks,” In Lecture Notes on Computer Sciences: Mechanisms for Autonomous Management of Networks and Services, vol. 6155, pp. 163–166, Springer, 2010

[10] G. Rueff, C. Thuen, and J. Davidson, “Sophia Proof of Concept Report,” Idaho National Laboratory (INL), 2010.

[11] Y. Zhang, L. Wang, W. Sum, I. Green, M. Alam, and others, “Distributed IDS in a multi-layer network architecture of smart grids,” IEEE Transactions on Smart Grid, num. 99, page 1, 2011

13

74

1099

1100

1101

1102

110311041105

110611071108

110911101111

1112111311141115

11161117

111811191120

112111221123

11241125

112611271128

11291130

113111321133

[12] P. Jokar, H. Nicanfar, and V. Leung, “Specification-based IDS for home area networks in smart grids,” in Proceedings of the IEEE International Conference on Smart Grid Communication (SmartGridComm), pp. 208—213, 2011

[13] R. Berthier and W. H. Sanders, “Specification-based IDS for AMI,” in Proceedings of the 17th Pacific Rim International Symposium on Dependable Computing (PRDC), pp. 184—193, 2011

[14] NIST SP 800-94, “Guide on Intrusion Detection and Prevention Systems (IDPS),” 2009

5.2 AcronymsAMI: Advanced Metering InfrastructureAMI-SEC: AMI securityANSI: American National Standards InstituteASAP-SG: Advanced Security Acceleration Project for the Smart GridCBKE: certificate-based key exchangeCSV: comma-separated valueC12.18 standard: ANSI standard for type 2 optical portC12.22 standard: ANSI specification for interfacing to data communication networksDPA: differential power analysisDMZ: demilitarized zoneHAN: home area networkHTTP: Hypertext Transfer ProtocolIDS: intrusion detection systemIEC: International Electrotechnical CommissionIETF: Internet Engineering Task ForceIT: information technologyMAC: media access controlMDMS: meter data management systemMIB: management information basesNAN: neighborhood area networksNESCOR: National Electric Sector Cybersecurity Organization ResourcesNIST: National Institute of Standards and TechnologyNIST CSWG: NIST Cyber Security Working GroupPDU: protocol data unitRFLAN: RF local area networkSGIP-CSWG: Smart Grid Interoperability Panel Cyber Security Working GroupSIEM: Security Information and Event ManagementSOAP: Simple Object Access ProtocolSPA: simple power analysisSYSLOG: IETF standard for computer data loggingWG: working groupXML: Extensible Markup LanguageZigBee: HAN communication protocol

113411351136

113711381139

1140

1141

1142

11431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177

AMI Cyber Security Incident Guidelines

Product ID Number

15

75

1178

1179

1180

1181

1182

1183

Advanced Metering Infrastructure Cyber Security Incident Guidelines

Product ID Number

Technical Update, August 10, 2012


1184

1185

1186

1187

1188

1189

1190

1191

1192

76






Southwest Research Institute (SwRI)

Automation and Data Systems Division

6220 Culebra Road, Building 68

San Antonio, TX 78238-5166, USA





77

1193

1194119511961197

119811991200120112021203

12041205120612071208

12091210

1211121212131214121512161217121812191220122112221223

1224

12251226

12271228

1229

78


Southwest Research Institute6220 Culebra RoadSan Antonio, TX 78238-5166

Principal Investigators

Tam Do; Gary Ragsdale, Ph.D., P.E.; Will Arensman

This report describes research sponsored by EPRI.



1230

12311232

123312341235

1236

1237

1238

1239

7980

81

82

ABSTRACTThis document is intended to be used by system and asset owners to assist in the preparation and response to AMI cyber security incidents. This document was developed by conducting interviews with EPRI members, AMI asset owners, and vendors, regarding practices involved in responding to AMI cyber security incidents and mapping the responses to requirements put forth by the Department of Homeland Security (DHS), National Institute of Standards and Technology (NIST), Open Smart Grid (Open-SG) Working Group, and Advanced Metering Infrastructure Security (AMI-SEC) working group.

KeywordsCyber Security, Incident Response, Best Practices

v

1240

1241124212431244124512461247

1248

1249

1250

83

CONTENTS

1 INTRODUCTION..................................................................................................................5-11.1 Usage........................................................................................................................5-11.2 References................................................................................................................5-2

2 REFERENCES AND BIBLIOGRAPHIES............................................................................2-12.1 List of References.....................................................................................................2-12.2 Definition of Terms....................................................................................................2-1

3 AMI INCIDENT RESPONSE PREPARATION GUIDELINES..............................................3-13.1 Incident Response Organization...............................................................................3-1

3.1.1 AMI Entity Identification and Oversight [9 – SG.IR-1].........................................3-13.1.2 Incident Response Program [9 – SG.IR-1].........................................................3-23.1.3 AMI Incident Response Development Facility [9 – SG.IR-3, SG.IR-4]...............3-33.1.4 Incident Response Roles and Responsibilities [9 – SG.IR-2].............................3-3

3.2 Incident Response Planning.....................................................................................3-43.2.1 Defining AMI Cyber Incident Response Requirements [3 – 4.1; 9 – SG.IR-1, SG.AT-1].........................................................................................................................3-43.2.2 Incident Scenarios Identification [9 – SG.IR-1, SG.AT-1]...................................3-53.2.3 Establishing a Continuity of Operations Plan [1 - 2.12.2; 9 – SG.CP-2].............3-63.2.4 Identifying Roles and Responsibilities for Continuity of Operations [9 – SG.AT-6]

3-73.3 Risk and Impact Assessment....................................................................................3-8

3.3.1 Identifying Internal Objectives and Functions.....................................................3-83.3.2 AMI Incident Classification [3 - 2]........................................................................3-93.3.3 Impact Ranking of AMI Functions [4 - 2.3]........................................................3-103.3.4 AMI Incident Impact Classification [4 - 2.3].......................................................3-113.3.5 AMI Incident Impact Analysis [4 -2.3]................................................................3-12

3.4 Incident Response Training....................................................................................3-153.4.1 Incident Process Training [1-2.7.5; 3-2.12.1; 9 - SG.AT-6]...............................3-153.4.2 Process Verification Testing [1-2.7.6]...............................................................3-15

4 AMI INCIDENT RESPONSE BEST PRACTICES GUIDELINES.........................................4-14.1 Incident Management Process [9 – SG.IR-5]...........................................................4-1

4.1.1 Incident Identification..........................................................................................4-24.1.2 Incident Remediation..........................................................................................4-64.1.3 Incident Review...................................................................................................4-8

5 CONCLUSION......................................................................................................................5-1

APPENDIX A: HISTORY OF AMI CYBER SECURITY INCIDENTS

APPENDIX B: CATALOG OF AMI INCIDENT ARTIFACTS AND FORENSICS

APPENDIX C: DESCRIPTIONS OF AMI INCIDENT SCENARIOS

vii

84

1251

1252

125312541255

125612571258

1259126012611262126312641265126612671268126912701271127212731274127512761277127812791280

12811282128312841285

1286

1287

1288

1289

1290

85

viii

86

1291

LIST OF FIGURES

Figure 1 – AMI Incident Impact Analysis Flowchart.................................................................3-13Figure 2 – AMI Incident Management Process..........................................................................4-2

LIST OF TABLES

Table 1 – Scope of Impact.......................................................................................................3-10Table 2 - AMI Incident Impact Classification............................................................................3-11Table 3 – Likelihood of Occurrence..........................................................................................3-13Table 4 – Risk Rating...............................................................................................................3-14Table 5 – AMI Incident Type....................................................................................................3-14Table 6 – Incident Scenario Description.....................................................................................4-3Table 7 – Physical Incident Scenarios.......................................................................................4-3Table 8 – Authentication Incident Scenarios..............................................................................4-4Table 9 – Communications Incident Scenarios..........................................................................4-5Table 10 – Software Incident Scenarios.....................................................................................4-6Table 11 – Incident Remediation Strategies..............................................................................4-7

ix

87

1292

1293

12941295

1296

1297

1298

12991300130113021303130413051306130713081309

1310

1311

88

IntroductionThe following document provides cyber security incident response guidelines and best practice recommendations specific to AMI systems. The document assumes that incident response guidelines and practices already exist for enterprise systems. The guidelines and practices described are specific to the unique components comprising the AMI system, hereafter referred to as the AMI logical architecture as defined in [3 – 4.2].

AMI logical architectures are different from other energy-related system architectures in several regards. The AMI logical architecture exists within the distribution grid and is closely associated with the utility’s customers. As the AMI is concerned with energy distribution and not transmission it does not fall under the scope of NERC CIP, ISO, or other bulk energy controls. Instead the AMI is more subject to mass media and customer scrutiny. The AMI also differs from other energy-related systems in that major components of the AMI such as the AMI meter operate outside the fence lines of utilities in areas that are easily physical accessible as compared to traditional electrical distribution grid components (e.g., substations and transformers).. . In some instances, AMI meters and web portals interact directly with customer-provided appliances and energy management systems using well-understood protocols. This document is dedicated to addressing these differences to the extent that the differences relate to AMI incident response. The document does not address incident response within the AMI-associated external to the AMI logical architecture as described in [3 – 4.2].

It is also assumed that the reader has access to and will become familiar with applicable standards cited in this document. The standards are included by citation only. The reader is encouraged to reference and become familiar with standards applicable to AMI systems.

5.3 UsageThis format of this document closely follows that of the NIST Special Publication (SP) 800-53, Information Security document [5] and the “U.S. Department of Homeland Security, Catalog of Control Systems Security: Recommendations for Standards Developers” document [1]. This document adopts the convention of adding a “supplemental guidance” to each requirement from the ASAP-SG Security Profile 2.0 document [3]. In this document, the supplemental guidance section contains the identified industry best practices for implementing the requirement.

Requirements are listed in the following format:

Requirement Titleo Descriptiono Requiremento Supplemental Guidanceo Requirements Enhancement

Rationale

This document is divided into two major sections: “AMI Incident Response Preparation Guidelines” and “AMI Incident Response Best Practices Guidelines.”

The goal of the “AMI Incident Response Preparation Guidelines” section is to lay the organizational groundwork for developing policies to respond to Advanced Metering Infrastructure (AMI) cyber security incidents.

1

89

13121313131413151316

1318131913201321132213231324132513261327132813291330

133113321333

1334

1335

133613371338133913401341

1342

13431344

1345

1346

1347

1348

13491350

135113521353

90

The goal of the “AMI Incident Response Best Practices Guidelines” section is to provide guidance on how to best respond to identified AMI cyber security threats.

5.4 ReferencesReferences throughout this document adopt the following format [<Reference #> - <Section #>].

<Reference #> - Refers to the document source listed in the reference section <Section #> - Refers to the specific section within the reference source

The absence of a <Section #> within the citation implicitly refers to the document in its entirety.

2

91

13541355

1356

1357

1358

13591360136113621363

6REFERENCES AND BIBLIOGRAPHIES6.1 List of References

1. U.S. Department of Homeland Security, Catalog of Control Systems Security: Recommendations for Standards Developers, April 2011.

2. U.S. Department of Commerce. National Institute of Standards and Technology, Computer Security Incident Handling Guide, Special Publication 800-61, Revision 1.

3. The Advanced Security Acceleration Project (ASAP-SG) - Security Profile For Advanced Metering Infrastructure, Version 2.0, June 22, 2010.

4. AMI-SEC (Advanced Metering Infrastructure Security) Task Force - AMI Risk Assessment (draft document), AMI-SEC Risk Assessment v0.9a-20080319.doc.

5. NIST Special Publication (SP) 800-53, Information Security, Revision 3, Updated 5/1/2010, http://csrc.nist.gov/publications/nistpubs/800-53-Rev3/sp800-53-rev3-final_updated-errata_05-01-2010.pdf .

6. UCAIUG: AMI‐SEC-ASAP AMI System Security Requirements V1.01, 12/17/20087. EPRI IECSA Volume II, Final Release, EPRI Use Case Repository, EPRI.com8. AMI Network V 3.0.doc, EPRI Use Case Repository, EPRI.com9. The Smart Grid Interoperability Panel (SGIP) – Cyber Security Working Group (CSWG),

NISTIR 7628 – Guidelines for Smart Grid Cyber Security: Vol. 1, Smart Grid Cyber Security Strategy, Architecture and High-Level Requirements, August 2010

6.2 Definition of TermsAMI entity –the owner and/or operator of the AMI logical architecture (e.g., cooperatives, municipal agency, or a corporate body).

Use case – a set of related activities and communications that render value in support of an AMI objective.

AMI function – a set of use cases supporting the achievement of a common AMI objective (e.g., billing).

AMI component – a subdivision of the AMI logical architecture primarily involved in one or more AMI use case communications or sequence of activities. AMI components may secondarily participate in non-AMI use cases. AMI components are described in [3- 4.2] .

AMI system – synonymous with AMI internal architecture are described in [3- 4.2].

AMI-associated components – are subdivisions of the AMI logical architecture external to the AMI internal architecture (e.g., Outage Management System, Customer Information System, and Distribution Automation System). The AMI associated components are depicted as the AMI logical architecture external perspective in [3- 4.2].

AMI logical architecture [3- 4.2] –In this document when the term AMI logical architecture is used, we are referring to the AMI logical architecture internal perspective. The components which are encompassed within the AMI logical architecture internal perspective are:

AMI Meters1

92

1364

1365

1366

13671368

13691370

13711372

13731374

137513761377

1378

1379

1380

138113821383

1384

13851386

13871388

13891390

139113921393

1394

1395139613971398

139914001401

1402

93

Field Tools

AMI Communications Network Devices

AMI Head End

Meter Data Management System

AMI Network Management System

AMI Meter Management System

Level of effort – is a measure of resources required to achieve an end (e.g., successfully attack an Advanced Metering Infrastructure (AMI) asset. Resources may include combinations of time, skill, money, and access to technology).

Loss of Control – The inability to exercise AMI control functions (e.g., remote disconnect)

Loss of Communications – The inability to communicate with the AMI (e.g., send/receive commands from a meter)

Loss of Trust/Integrity – The inability to verify the security of functions or data within the AMI (e.g., compromise of keys, integrity check failures, billing data compromise)

Loss of Privacy – The inability to protect the privacy of information (e.g., compromise of keys, loss of encryption functionality)

Loss of Attribution – The inability to attribute an AMI action (e.g., meter log on) to its actor

Natural Event – An event which can be correlated with other situational awareness data (e.g., weather forecast, fires, earthquakes). These events are generally caused by an act of nature.

Man-Made Event – An event not caused by nature and is generally caused as a result of a man-made event.

Malicious Event – An event which has been caused with ill-intent. It generally occurs in the absence of natural or unintentional events. These events are identified by correlating the event with other situational awareness data (e.g., meter tampers, meter authentication attempts). Depending on the severity of the event, the AMI entity may need to notify law enforcement (e.g., local, federal).Types of events malicious events include:

AMI cyber attack

Civil unrest

Vandalism

Energy Theft

Non-Malicious Event – An event which has been caused unintentionally, or as the result of operator or system design error. These events generally arise as a result of AMI operation, misconfiguration, or design errors.

Objective – a level of achievement necessary for a desired outcome or state of AMI entity business activity (e.g., customer communication).

2

94

1403

1404

1405

1406

1407

1408

140914101411

1412

14131414

14151416

14171418

1419

14201421

14221423

14241425142614271428

1429

1430

1431

1432

143314341435

14361437

3

95

1438

1439

96

AMI Incident Response Preparation GuidelinesThe purpose of Section 3 and its subsections are to define the preparation guidelines for planning the incident response procedures defined in Section 4. The guidelines provide the organizational framework upon which incident response procedures are rationally created, validated, and justified.

For this section a subset of guidelines has been selected which pertain specifically to the unique aspects of the Advanced Metering Infrastructure (AMI) such as:

AMI meters operate in geographically distributed and easily physically accessible locations (e.g., homes, apartment buildings, and strip malls)

AMI meters may contain remote connect/disconnect capabilities

AMI meter data may be used in organizational demand response programs therefore the integrity of the data is of concern

AMI systems often contain mesh communication networks allowing for the potential to leverage the network to attack a large number of meters

The guidelines in this section should be considered as a supplement to any existing organizational incident response plan, and should be used to augment incident response approaches as applicable to AMI.

This section is divided into four subsections: incident response organization, incident response planning, risk and impact assessment, and training.

6.3 Incident Response OrganizationThis subsection describes the requirements and provides guidance on how to organize an AMI incident response plan.

6.3.1 AMI Entity Identification and Oversight [9 – SG.IR-1]The responsibility and accountability for AMI incidents fall to AMI entity having responsibility and authority over an AMI logical architecture.

An individual or a group of individuals responsible for the different domains within the AMI entity should be identified and charged with leading the development of the AMI incident response program. It is important for this individual leader, or collective group of leaders to take responsibility for the planning and implementation of cyber security incident response processes and procedures.

6.3.1.1 Requirement

An individual or a collective group of individuals with authority that encompasses all aspects of the AMI logical architecture should oversee the AMI incident response team and process.

Organizations may subdivide leadership of the AMI into different operational domains. Examples of operational domains are:

Business Processes

o Responsible for handling the business operations of AMI logical architecture such as revenue assurance and billing.

AMI Operations

1

97

1440144114421443

14451446

14471448

1449

14501451

14521453

145414551456

14571458

1459

14601461

1462

14631464

14651466146714681469

1470

14711472

14731474

1475

14761477

1478

98

o Responsible for day-to-day operation of the AMI internal architecture such as maintenance and implementation of the meter data management system.

Information Technology (IT) Operations

o Responsible for overall management of the AMI associated components such as servers, backend appliances, and the organization’s security information and event management system (SIEM).

For each identified operational domain within the organization, a leader should be identified to coordinate the incident response process as it pertains to their domain.

6.3.1.2 Supplemental Guidance

The designated leader within the AMI entity directs the formation of the incident response capability and governs the administration of the capability once it is operational. The AMI incident response designated leader has approval authority for incident plans, policies, and procedures [1-2.2, 2.3].

6.3.1.3 Requirements Enhancement

None

6.3.2 Incident Response Program [9 – SG.IR-1]The AMI logical architecture realizes use cases, thereby achieving business objectives specific to the automated collection of meter data, provisioning of power to a customer, and ensuring the integrity of the entity-customer relationship. AMI-specific use cases must be realized in a timely, reliable, and secure fashion for the relationship to prosper and entity business objectives to be achieved.

The purpose of the AMI incident response program is to sustain or restore the AMI use cases despite incidents that would otherwise disrupt the AMI logical architecture. An AMI incident response program addresses the unique cyber security risks with a corresponding capability to continue or resume operations of an Advanced Metering Infrastructure (AMI) in the event of disruption of AMI use cases and objectives.

The AMI incident response program entails the preparation, testing, and maintenance of AMI-specific policies, procedures, processes, systems, and skill sets. The program enables the AMI entity to restore the AMI’s operational status after the occurrence of a disruption. Disruptions can come from natural disasters, such as earthquakes, tornados, floods, or from manmade events like riots, terrorism, or vandalism. The ability for the AMI to function after such an event is directly dependent on implementing program prior to incidents using the organization’s planning process. [1-2.12].

6.3.2.1 Requirement

The AMI entity establishes an AMI incident response program as an identifiable and recognized organizational function.


The official establishment and visibility of the AMI incident response program as a formal and unique activity is important to its adoption and support within the AMI entity.

2

99

14791480

1481

148214831484

14851486

1487

1488148914901491

1492

1493

1494

14951496149714981499

15001501150215031504

1505150615071508150915101511

1512

15131514

1515

15161517


None

6.3.3 AMI Incident Response Development Facility [9 – SG.IR-3, SG.IR-4]AMI logical architectures are complex internal and associated components satisfying AMI entity goals including reliability, integrity, and functionality. Like other systems, it is desirable to develop and verify incident response programs without risk to an operational AMI logical architecture. Therefore, it is good practice to use an AMI incident response development facility separate from the production AMI system.

6.3.3.1 Requirement

A facility exists as a development and test vehicle for the creation, maintenance, and verification of AMI incident response policies, procedures, training, processes, and components.


AMI systems have distinct differences from enterprise systems often found within an AMI entity and compose of systems such as:

Electric Meters

RF, Wireless, Wired and Cellular Communications Networks

Signing and Encryption Appliances

Meter Data Management Systems (MDMS)

Additionally, many components of the AMI system operate in potentially harsh and physically unsecured environments.

The facility should to the extent possible replicate the unique aspects of the AMI system and its environment. The facility may consist of functional test beds used in development to test configurations of meters and systems prior to wide-scale deployments.


None

6.3.4 Incident Response Roles and Responsibilities [9 – SG.IR-2]The skill sets required to develop, test, and execute an AMI incident response program are many and in many cases unique to the AMI logical architecture. Likewise, the duties to be performed within the program are often unique to the AMI system. For example, the understanding of the AMI meter and its communications networks (e.g., wide-area and neighborhood area) is a skill set different from a typical IP-based enterprise system. The designated roles to respond to a meter intrusion or an AMI network denial of service attack are also different as a meter technician may require training in non-IP communication protocols, AMI hardware configuration parameters, AMI diagnostic techniques, and cyber-security methods.

6.3.4.1 Requirement

The duties, skill sets, and responsibilities of roles within the AMI incident response program are clearly defined and communicated within the AMI entity.

3

100

1518

1519

1520

15211522152315241525

1526

15271528

1529

15301531

1532

1533

1534

1535

15361537

153815391540

1541

1542

1543

15441545154615471548154915501551

1552

15531554

101


Each organization may sub-divide its AMI leadership into different functional domains. The AMI incident response team should consist of members from each of these functional domains and should include representatives from domains relating to the AMI-associated components..

The following groups should be considered in the formation of a cross-functional team that plans for and responds to AMI incidents:

Business (e.g., Legal, Accounting, Human Resources, Public Relations)

o Charged with responding to the business impacts of the AMI incident.

AMI Operations

o Charged with responding to the operational impacts of the AMI incident.

Information Technology (IT) Operations

o Charged with responding to the impacts of the AMI incident as it relates to the IT system.

The cross-functional team may be adjusted depending on the scope of impact of the AMI incident.

6.3.4.3 Requirement Enhancement

The AMI framework is an integral part of the larger organizational framework. The organizational framework includes AMI incident response roles with well-defined skills, accountability and responsibilities.

6.4 Incident Response PlanningThis section provides requirements and guidance on how to plan and prepare for an AMI incident.

6.4.1 Defining AMI Cyber Incident Response Requirements [3 – 4.1; 9 – SG.IR-1, SG.AT-1]

The AMI system is implemented to satisfy a set of AMI-specific use cases and derive business value. Typical use cases can be found within the Advanced Security Acceleration Project (ASAP-SG) - Security Profile for Advanced Metering Infrastructure, Version 2.0, document [3– 4.1]. The document provides traceability of AMI requirements to business use cases and associated business objectives supported by the AMI system.

The goal of the AMI incident response program is to satisfy AMI requirements associated with data integrity, reliability, timeliness, privacy, and accessibility when incidents disrupt the realization of AMI use cases. A thorough understanding of AMI requirements is necessary for the proper formulation of policies, procedures, training, and resources comprising the AMI incident response program.

6.4.1.1 Requirement

AMI incident response program satisfies as set of clearly defined and articulated requirements traceable to the AMI system use cases and business objectives.

4

102

1555

155615571558

15591560

1561

1562

1563

1564

1565

15661567

15681569

1570

157115721573

1574

15751576

15771578

15791580158115821583

15841585158615871588

1589

15901591


The AMI use cases produce a set of business objectives when successfully achieved or a set of business impacts when disrupted by an incident. AMI incident response requirements should be considered in light of the business objectives and impacts associated with a given incident scenario.

6.4.1.3 Requirement EnhancementsNone

6.4.2 Incident Scenarios Identification [9 – SG.IR-1, SG.AT-1]A proactive stance toward AMI incidents requires the AMI program should anticipate incidents before they are encountered. The incident scenario analysis considers the impacts upon use cases and business objectives caused by benign or malicious attacks on AMI meters, communication devices, head end, forecasting system, meter management system, network, management system, and other AMI components as defined in [3- 4.3].

6.4.2.1 Requirement

The AMI incident response program identifies AMI incident scenarios and the possible causes for those AMI incidents. The incident scenarios should identify forensic evidence required for detection the AMI incident. The AMI incident response program should anticipate AMI incident scenarios which may not yet occurred.


To proactively identify AMI incidents it is important that the organization develop an overall AMI incident response plan that provides a process for responding to AMI security incidents regarding vulnerabilities which have been identified and have not yet been remediated, as well as vulnerabilities for which the organization has accepted as a risk.

The AMI incident response plan should include methods for assigning incident impact and asset-value metrics (e.g., monetary, or objective-oriented).

The AMI incident plan should include practices for:

Reviewing and updating incident response procedures.

How to respond to newly released vulnerabilities.

Metrics for determining the impact of newly identified vulnerabilities.

Methods for determining the type of AMI incident (e.g., malicious, non-malicious, natural)

The risk analysis described in Section 6.5.5 should be applied to scenarios after the plausibility of the scenario has been adequately established through examination or actual occurrence.

Additionally, the AMI incident response plan should include:

Risk mitigation strategies

Policies and procedures for responding to incidents based on the impact and type of AMI incident

5

103

1592

1593159415951596

1597

1598

1599

16001601160216031604

1605

1606160716081609

1610

1611161216131614

16151616

1617

1618

1619

1620

16211622

16231624

1625

1626

16271628

104

6.4.2.3 Requirement Enhancement

None

6.4.3 Establishing a Continuity of Operations Plan [1 - 2.12.2; 9 – SG.CP-2]A continuity of operations plan ensures the resumptions of services in the event of an AMI incident.

6.4.3.1 Requirement

The organization should develop, test and implement a continuity of operations plan dealing with the overall issue of maintaining or re-establishing operation of the AMI system in case of an undesirable interruption. The plan should addresses roles, responsibilities, assigned individuals with contact information, and activities associated with restoring system operations after a disruption or failure. Designated officials within the organization review and approve the continuity of operations plan. Individuals responsible for the systems covered by the continuity of operations plan should be trained on the implementation and execution of the continuity of operations plan.


A continuity of operations plan addresses both business continuity planning and recovery of all vital AMI system operations.


The continuity of operations plan may include:

Verification of communications with all AMI meters, collectors, and relays

The restoration of disconnect switches to known states

Verification of the integrity of meter firmware

Verification of the integrity of meter data

The continuity of operations plan should also address:

Unintentional AMI cyber security incidents (e.g., operator error, accidents)

Natural events (e.g., high velocity winds, flooding, earthquakes)

Intentional incidents as a result of:

o Disgruntled current and past employees

o Hackers

6.4.3.3.1 Rationale

Experience demonstrates that systems fail and mistakes occur. An AMI continuity of operations plans allow for an orderly recovery from such situations. Critical analysis of a system often brings weaknesses to light, thereby giving the AMI incident response program opportunities to design additional safeguards into the AMI system necessary for and orderly recovery.

6

105

1629

1630

1631

16321633

1634

16351636163716381639164016411642

1643

16441645

1646

1647

1648

1649

1650

1651

1652

1653

1654

1655

1656

1657

1658

1659166016611662

6.4.4 Identifying Roles and Responsibilities for Continuity of Operations [9 – SG.AT-6]

The organization’s continuity of operations plan should define and communicate the specific roles and responsibilities for each part of the plan in relation to various types of disruptions to the operation of the AMI system.

6.4.4.1 Requirement

The organization’s continuity of operations plan defines and communicates the specific roles and responsibilities for each part of the plan in relation to various types of AMI incidents. [1 – 2.12.3].


An AMI incident continuity of operations plan defines the roles and responsibilities of the various employees and contractors in the event of an incident. The plan identifies responsible personnel to lead the recovery and response effort if an incident occurs.

The following roles and responsibilities are representative of the organizational domains involved with AMI and as a result they should be considered within the continuity of operations plan:

Incident Coordinatoro Responsible for organizing, dispatching and coordinating the incident response team o Trained in the incident response policies, procedures, roles, responsibilities, and goals

AMI Operations Leado Responsible for the day-to-day operations of AMI system within the organizationo Assists in identifying key personnel within the AMI operations group to assist in the

incident response processo Leads efforts to quantify and categorize incident impactso Trained in the AMI functions, use cases, goals, metrics, and incident response procedures

IT Operations Leado Responsible for the day-to-day operations of information systems within the organizationo Assists in identifying key personnel within the IT operations group to assist in the

incident response processo Assists the AMI Operations lead in the quantification and categorization of incident

impactso Trained in information technology and enterprise cyber security.

Business Operations Leado Responsible for the day-to-day business operations within the organizationo Assists in contacting the affected business entities within the organization o Assists the AMI lead in the quantification and categorization of incident impactso Leads the mapping of the AMI entity's organizational objectives into cyber security

metrics o Trained in the AMI entity’s planning and budgeting

Public Relations Lead

7

106

16631664

166516661667

1668

166916701671

1672

167316741675

167616771678

16791680

1681

16821683

168416851686

1687

16881689

16901691169216931694

16951696

1697

1698

169917001701

1702

107

o Responsible for correspondence with the media, regulators, and other external groups.o Trained in the legal entity’s external communications, reporting, and marketing policies

and practices. Legal Lead

o Responsible for liaising with law enforcement and regulatory agencies regarding planning and responding to AMI cyber security incidents.

o Trained in judicial evidentiary data collection and custodial control requirements, external reporting procedures and policies, and regulation governing cyber security incidents.

These generic categories were compiled from the interviews conducted with EPRI members, AMI asset owners, and vendors, regarding the practices involved for responding to AMI cyber security incidents. Each organization may have different titles and categories for these roles and should make their best determination in mapping the provided categories to their organization.


None

6.4.4.3.1 Rationale

Specific roles and responsibilities within continuity plans help alleviate confusion in the event of a significant incident. This clarification can ease the process and better focus the organization when recovering from a disruption.

6.5 Risk and Impact AssessmentThis section provides requirements and guidance on how to perform a risk and impact assessment with the goal of assisting the AMI entity in prioritizing incident response efforts.

6.5.1 Identifying Internal Objectives and FunctionsAMI entity objectives should be considered as part of the risk and impact assessment. Methods and use cases for those objectives should identify the tangible and intangible benefits to the AMI entity.

6.5.1.1 Requirements

The organization identifies AMI entity objectives and the use cases implemented to achieve those goals.


An AMI entity should identify internal objectives and functions that would be affected by an AMI incident and should consider those objectives when assigning a risk and impact rating to the identified incidents. Examples of AMI entity internal objectives that may be affected by AMI incidents are:

Real-Time Pricing (RTP) – Top Level [7 – D19] Remote Connect/Disconnect [6– 2.1.1] Demand Response – Utility Commanded Load Control [7 – D12]

8

108

1703

170417051706170717081709171017111712

1713171417151716

1717

1718

1719

172017211722

1723

17241725

1726

172717281729

1730

17311732

1733

1734173517361737

173817391740

AMI Network (Moving Data Elements from the AMI Head-End to Smart Meter & from the Smart Meter to the AMI Head-End) [8]

Firmware Updates [9 – 2.3.17]


AMI entity objectives and use cases are impacted by AMI incidents in varying degrees. Use cases affected by AMI cyber-security incidents are identified and associated with organizational objectives.

6.5.2 AMI Incident Classification [3 - 2]The AMI program should define common methods, vernacular, and criteria for describing a given AMI incident. Incidents may be classified according to common incident characteristics (e.g., failure mechanisms, mitigation methods, resource requirements, impact assessment ranking). Categorizing incidents facilitates better planning, organization, and uniformity within the incident response program.

6.5.2.1 Requirement

AMI incidents are subdivided into categories and classified according scope of impact, likelihood of occurrence and incident type.


The following criteria should be used to describe AMI incidents:

Scope of Impact

A classification based on the number of systems affected by the AMI incident. Scope of impact ratings can be used to determine an incident’s severity and impact on the operation of the AMI logical architecture.

Likelihood of Occurrence:

A classification based on the current threat environment of how likely the AMI incident is to occur. Likelihood of occurrence ratings can be used to reprioritize responses to AMI incidents.

Type:

A classification based on the intent of the perpetrator of the AMI incident (e.g., natural, man-made, malicious, or non-malicious).


None

6.5.3 Impact Ranking of AMI Functions [4 - 2.3]The AMI program should assess the level of AMI entity concern associated with the loss of an AMI function.

6.5.3.1 Requirement

AMI incident impact rankings are assigned to assets, functions, or use cases according to the consequence and severity of an incident that disrupts portions or all of the functional capability.

9

109

174117421743

1744

174517461747

1748

17491750175117521753

1754

17551756

1757

1758

1759

176017611762

1763

176417651766

1767

17681769

1770

1771

1772

17731774

1775

17761777

110


AMI incident impact rankings should be developed through threat assessment exercises. The threat assessment exercise should enumerate all components of the AMI logical architecture, enumerate the components’ interactions with external systems, assess the potential vulnerabilities within each component and component interface, and evaluate the impact (e.g., organizational and operational) of the vulnerability being exercised.

Example impact ranking definitions are provided below in Table 15.Table 15 – Scope of Impact

Scope of Impact

1 Low

The incident has a low impact limited to a single unit (e.g., Electric Meter). Measures should be taken to address the threat related to the single unit.

2 Medium

The incident has a medium impact limited to units in a single grouping of AMI logical architecture components (e.g., a meter cell). Measures should be taken to identify the root cause of the threat and remediate the threat across all affected systems.

3 High

The incident has a high impact and the effects span the entire AMI logical architecture (e.g., meter, routers, head end). Measures should be taken to immediately contain the threat and a cross-functional team of AMI operators, IT system operators, AMI vendor, and business representatives, should be formed to remediate the threat.

The scope of impact ratings listed here is provided as an example. Scope of impact for vulnerabilities should be determined on a case-by-case basis depending on the function and number of devices impacted.


None

6.5.4 AMI Incident Impact Classification [4 - 2.3]The AMI incident response program should classify AMI incidents based upon the impact of the AMI incident to AMI entity objectives.

6.5.4.1 Requirement

Incidents are assigned an impact classification according to the AMI entity objective impacted by an incident.


Table 16 provides an example mapping of AMI incidents to impact classifications based upon incident’s scope of impact. It is recommended that the reader formulate their own classification of AMI incidents and tailor it based on the unique aspects of their AMI deployment (e.g., usage of C12.18 keys) and their organization’s AMI objectives.

10

111

1778

17791780178117821783

1784

1785

178617871788

1789

1790

1791

17921793

1794

17951796

1797

1798179918001801

Table 16 - AMI Incident Impact Classification

ID AMI Incident Classification

Description Impact Classification

1 Theft of Energy An attacker attempts to steal energy by modifying the meter and/or the metering data.

Low – The attack is associated with a single occurrence of energy theft and has negligible impact.

Medium – The attack is associated with multiple occurrences of energy theft and has a direct monetary impact.

High – The attack is associated with a large scale theft of energy and can have both monetary and functional impacts

2 Theft of C12.18 Logon Credentials

An attacker has stolen the C12.18 login credentials from a single meter.

Low – The C12.18 credentials are different for each meter and have a minimal impact.

Medium – The C12.18 credentials are by multiple meters but are limited due to the physical requirement for exploitation.

High – The C12.18 credentials are in use by multiple meters and also have a high scope of impact due to the ability to perform the attack wirelessly or remotely.

3 Bypass of Security Controls

An attacker has bypassed security controls on the meter.

Low – The bypass of security credentials only affects a single device at a time and has a limited impact.

Medium – The bypass of security controls affects multiple devices but has limited impact in terms of control.

High – The bypass of security controls affects multiple devices and allows full bypass of security controls on the affected devices.

4 Failed Communications Integrity

An integrity check failure has occurred on the meter.

Low – The integrity check failure occurs in a limited number of instances and affects a limited number of meters.

Medium – The integrity check failure occurs on a large scale and may affect the quality of data being received from the meters.

High – The integrity check failure occurs across all data being received from the meters.

Thresholds on number of meters affected should be defined by each organization to rank the impact of the incident.

5 Power Outage An AMI incident results in a power outage.

Low – A limited number of customers are affected (e.g., single household or business).

Medium – Multiple customers are affected by

11

112

1802

113

the effects are limited and isolated to a specific geographical area (e.g., neighborhood).

High – A large scale power outage occurs affecting a significant portion of the AMI deployment (e.g., city or state). High impact outages may also include life and safety critical services (e.g., hospitals and emergency first responders).

6 Software Error An AMI incident results in a software error occurring on the meter.

Low – The software error occurs in a limited number of devices and has a low impact (e.g., causes a reset or reboot).

Medium – The software error occurs in multiple meters and has a high impact (e.g., toggling of disconnect switch, denial of service)

High – The software error occurs a significant portion of the AMI deployment (e.g., city or state) and has a high impact (e.g., toggling of disconnect of switch).


None

6.5.5 AMI Incident Impact Analysis [4 -2.3]Define the methods for assessing the value of the AMI function to an AMI entity in terms that can also be used to quantify incident impact if part or all of the AMI logical architecture are rendered ineffective.

6.5.5.1 Requirement

The AMI incident impact metrics assigned to the AMI incidents quantify incident response risk exposure.


The AMI entity should identify the AMI functions and use cases to be included in the incident impact analysis.

The following methodology may be adopted in assigning risk rankings to AMI assets.

12

114

1803

1804

1805

1806

180718081809

1810

18111812

1813

18141815

1816

1817

Figure 10 – AMI Incident Impact Analysis Flowchart

First an impact ranking is assigned to AMI assets using a methodology such as that defined in the Table.

Next an exercise is performed where AMI incidents are defined and categorized. AMI incidents are assigned a scope of impact ranking based on the number AMI assets they affect.

Finally a likelihood of occurrence rating is assigned to the AMI incident based on historical evidence of similar types of incidents. An example of likelihood of occurrence ratings to be used is provided below in Table 17. The likelihood of occurrence table provided is a simplified version based off of the AMI-SEC likelihood interpretation policy [4 – 2.6.1].

Table 17 – Likelihood of Occurrence

Likelihood of Occurrence

1 LowNot expected to occur, or may occur in exceptional circumstances.

2 Medium Could occur at some time.

3 High Will probably occur in most circumstances.

Once the AMI incidents have been defined, categorized and assigned an impact and likelihood of occurrence rating, a risk rating can be determined by using a matrix such as shown in Table 18.

Table 18 – Risk Rating

Risk Rating

Likelihood of Occurrence

Scope of Impact

Low Medium High

1 2 3

13

115

1818

1819

1820

18211822

18231824

1825182618271828

1829

18301831

1832

116

1 Low Low Medium Medium

2 Medium Low Medium High

3 High Low High High

The risk ratings help to guide planning, budgeting, and resource allocation processes to address the issues of highest concern and impact.

Additional metrics may be included in the risk rating in the event of an incident response such as the type of AMI incident.

The following type classification has been provided as an example:Table 19 – AMI Incident Type

Type

1 Malicious

The incident has been determined to have been caused by a malicious actor. Additional measures may be taken to contain the threat until the threat has been remediated.

2Non-Malicious

The incident has been determined to have been caused by a non-malicious actor such as an operational error. Depending on the severity of the error, measures may be taken to contain the incident impact.

3 Natural

The incident has been determined to have been caused by a natural event such as an earthquake, or storm. Normal operational procedures should be utilized to contain and remediate the incident.

Such ratings are dynamic and may not be able to be determined prior to an incident occurring; however it may be used in adjusting the incident response in the course of investigating an AMI incident.


None

6.6 Incident Response TrainingThis section provides requirements and guidance on how to develop and assess incident response training.

6.6.1 Incident Process Training [1-2.7.5; 3-2.12.1; 9 - SG.AT-6]The AMI incident response program should identify the types of training and materials required to prepare the AMI entity, its personnel, and those affected by AMI incidents

6.6.1.1 Requirement

The program includes training on the creation and implementation of the AMI incident response plans for employees, contractors, and stakeholders. The program provides refresher training

14

117

1833

18341835

18361837

1838

1839

1840

184118421843

1844

1845

1846

18471848

1849

18501851

1852

18531854

annually. The training covers employees, contractors, and stakeholders in the implementation of the continuity of operations plan.


Training needs are to be provided to individuals in the AMI community so that all understand the content, purpose, and implementation of the incident response plans. Different levels of incident response training may be prepared for personnel with different levels of AMI responsibility. Refer to 6.4.4 for the different AMI incident response roles and responsibilities and recommended training.


None

6.6.2 Process Verification Testing [1-2.7.6]Regular testing helps to determine the effectiveness of the AMI incident response plans.

6.6.2.1 Requirement

The program regularly tests AMI incident response plans to validate the efficacy of the program.


Following the preparation of the AMI incident response plans, a schedule is developed to review and test each of the plans to ensure that it continues to meet its AMI objectives.


None

15

118

18551856

1857

18581859186018611862

1863

1864

1865

1866

1867

1868

1869

18701871

1872

1873

1874

119

7AMI INCIDENT RESPONSE BEST PRACTICES GUIDELINESThis section focuses on the best practices for responding to AMI incidents. This section introduces the AMI incident management process and provides specific best practice guidance regarding incident detection, remediation and review.

7.1 Incident Management Process [9 – SG.IR-5]A phased approach can greatly streamline the incident notification and mobilization process. An example of one such approach is provided. The organization may tailor the approach to best fit their organization.

1) Incident Identification

a. Incident is identified through an automated process such as an SIEM or reported manually through another source

b. Evidence such as alarms, events and situational awareness data is collected and preserved

c. Potential incident scenarios are classified and identified based on available evidence and the incident response teams are notified

2) Remediation

a. If a remediation does not exist, the incident response team will develop a remediation strategy which may involve a root cause analysis of the incident

b. The applicable remediation strategy is applied based on the identified scenario

3) Review

a. If necessary, regression testing is performed in order to verify that the vulnerability or exploit has been remediated.

b. A review of the incident is performed in order to determine if the incident response strategy was effective and whether additional training for incident responders is necessary or if the incident detection rule sets need to be fine-tuned.

Figure 11 provides a detailed view of the AMI incident management process.

1

120

1875

1876

1877

187818791880

1881

188218831884

1885

18861887

18881889

18901891

1892

18931894

1895

1896

18971898

189919001901

1902

Figure 11 – AMI Incident Management Process

The following sections provide guidance and best practices for the incident management process.

7.1.1 Incident IdentificationIncident identification is the first step in the incident management process. Identification of AMI incidents involves the collection of forensic evidence (e.g., alarms, events and situation awareness data) in order to determine the incident scenario. Identification of AMI incidents and the incident scenario allows for the AMI entity to determine the appropriate next steps such as developing a remediation strategy or accepting the risk.

7.1.1.1 Security Information and Event Management Systems [9 – SG.IR-6]

Security Information and Event Management (SIEM) systems allow an organization to organize and correlate live situational awareness data from multiple systems, including non-AMI systems to detect security events or incidents.

7.1.1.1.1 Guidance

Organizations should develop AMI specific rule sets for their SIEM systems. SIEM rule sets may include situation awareness data to assist in the detection of AMI incident and incident scenarios.

The SIEM should be configured to collect alarms and events from all components of the AMI logical architecture.

Once the rule sets have been developed and the SIEM’s have been configured, the organization should fine tune the rule sets in such a way that they do not result in an unacceptable number of false positive AMI incident detections.

2

121

19031904

1905

1906

19071908190919101911

1912

1913

191419151916

1917

191819191920

19211922

192319241925

122

7.1.1.2 Data Mining

Data mining allows an organization to analyze archives of past security logs to identify intrusions.

7.1.1.2.1 Guidance

Organizations should archive, for an organizationally defined period of time, AMI-related SIEM alarms, events and situational awareness data. Data mining solutions may then be used to allow for a forensic investigation to detect signs of current or past intrusions and fine-tune AMI SIEM rule sets to detect intrusions and reduce the number of false positive AMI incident detections.

7.1.1.3 Log Message Integrity

Integrity checks on log messages can be used to determine if messages been tampered or modified from acquisition to analysis, transmission and storage. Additionally, integrity checks should be used to allow for preservation of forensic evidence for legal purposes.

7.1.1.3.1 Guidance

Organizations should utilize integrity checks such as cryptographic signatures to detect tamper or modification of log messages during transmission. Modification to data in transit may indicate a cyber-security intrusion. Organizations should also employ integrity checks to preserve SIEM data for forensic and legal purposes.

7.1.1.4 Incident Scenarios [9 – SG.IR-8]

This section provides guidance on how to identify specific AMI incidents. The following fields in Table 20 are used to describe each of the incident scenarios listed in the sections below. After the incident scenario has been determined, the incident classification field in Table 25 is used to determine the appropriate remediation strategy to employ.

Table 20 – Incident Scenario Description

ID Identifier for the incident scenarioIncident Scenario Description of the incident scenarioAlarms and Events Alarms and events associated with the incident scenarioSituation Awareness Data Evidence to support that the incident has occurredIncident Classification Classification of the incident to allow the incident response team

to determine an appropriate remediation strategy to employ

7.1.1.4.1 Physical

The following incident scenarios occur as a result of a physical action being taken upon the meter.

Table 21 – Physical Incident Scenarios

ID Incident Scenario Alarms and Events Situational Awareness Data Incident Classification

PHY.1 Attacker installs bypass device to

Removal Tamper A physical bypass device Theft of Energy

3

123

1926

19271928

1929

1930193119321933

1934

193519361937

1938

1939194019411942

1943

19441945194619471948

1949

1951

19521953

1954

steal electricity has been installed.

PHY.2 Attacker tampers with meter to steal electricity

Removal Tamper Tamper seals on the meter have been broken.

Signs of hardware modification present (e.g., wires or headers soldered onto board)

Theft of Energy

PHY.3 Attacker vandalizes meter to steal electricity

Removal Tamper

Communications Lost

Signs of physical damage to the meter.

Theft of Energy

PHY.4 A natural event (e.g., earthquake, fire, or flood) causes the meter to lose power.

Device Outage

Communications Lost

Weather or news reports may indicate that a natural event has occurred.

Power Outage

PHY.5 Attacker removes meter and inserts it backwards into the meter housing to steal electricity

Removal Tamper

Reverse Rotation

Physical inspection reveals meter is inserted backwards.

Theft of Energy

7.1.1.4.2 AuthenticationThe following incident scenarios occur as a result of an attacker attempting to bypass authentication controls on the meter.

Table 22 – Authentication Incident Scenarios


AUTH.1 Attacker attempts to guess the login credentials to the C12.18 optical port interface

C12.18 Failed Login

C12.18 Successful Login

Event occurs in the absence of any known maintenance activity

Theft of C12.18 Logon Credentials

AUTH.2 Attacker attempts to bypass the meter’s security controls by injecting C12.22 messages with invalid credentials

C12.22 Failed Authentication

Inspection of packet captures to detect maliciously modified packets

Bypass of Meter Security Controls

4

124

1955

1956195719581959

1960

19611962

125

7.1.1.4.3 CommunicationsThe following incident scenarios occur as a result of a disruption of communications between the meter and the head end.

Table 23 – Communications Incident Scenarios


COM.1 Poor communications between the AMI meter and the head end causes integrity check failures

Failed Communications Integrity Check

Failed communications integrity checks occur sporadically without any discernible pattern (e.g., types of messages failing integrity checks).

Poor Communications Link

COM.2 Attacker injects fuzzed packets on the communications link in order to bypass security controls


Malformed PDU

Events occur with a discernible pattern (e.g., types of messages failing, period in which events occur)

Bypass of Security Controls

COM.3 Attacker injects malformed packets in an attempt to spoof the meter’s energy usage

Failed Integrity Check Events occur with a discernible pattern related to billing or metering functions.

Theft of Electricity

COM.4 Attacker employs an active RF jamming attack to disrupt meter communications

Communications Lost A number of meters in the same physical locality report a loss of communications

Denial of Service

COM.5 Attacker floods the communications channel with traffic in order to disrupt meter communications

Communications Lost


Malformed PDU

Meter traffic flow monitoring tools report an abnormally large volume of traffic

Denial of Service

COM.6 Attacker exploits a meter vulnerability to cause a permanent denial of service

Communications Lost Event occurs in the absence of any additional situational awareness data (e.g., natural event, tamper events)

Denial of Service

7.1.1.4.4 SoftwareThe following incident scenarios occur as a result of a software error on the meter.

5

126

1963196419651966

1967

1968

196919701971

Table 24 – Software Incident Scenarios


LOG.1 Attacker injects malicious code in order to bypass security controls on the meter

Software Error

Malformed PDU

Additional signs of suspicious behavior (e.g., unauthorized remote disconnects)

Bypass of Security Controls

LOG.2 Meter software flaw causes software errors to occur

Software Error Incident occurs under specific circumstances in similarly configured meters

Meter Malfunction

7.1.2 Incident RemediationThe next step in the incident management process is remediation. Remediation of AMI incident involves identifying the appropriate remediation strategy to employ. In most circumstances, Remediation of AMI incidents should be performed after the AMI incident scenario and incident classification has been identified.

The following best practices have been identified for the remediation of AMI incidents.

7.1.2.1 Event Log Retention [9 – SG.IR-10]

Event logs allow for the forensic analysis of AMI incidents.

7.1.2.1.1 Guidance

Event logs (e.g., alarms, events, and situational awareness data) should be periodically archived, as defined by organizational policy, for the following purposes:

Data Mining

o Historical event log data can be used in conjunction with data mining methods to improve SIEM algorithms and detect specific AMI incidents.

Evidence in Court

o Historical event log data may be used to prosecute criminals in the court of law.

Due to the distributed nature of AMI electric meters, it is recommended that event logs be periodically retrieved from the meter and stored in a centralized or distributed system. The amount of time to store AMI event log data may be determined by corporate policies.

7.1.2.2 Developing a Remediation Strategy [9 – SG.IR-9]

The AMI entity should create incident classifications and develop remediation strategies in the event that the identified incident has occurred. Table 25 provides a list of incident classification and remediation strategies. The specific remediation strategy to employ will depend on the impact ranking of the AMI incident. An example of how to develop impact rankings is provided in Section 6.5.4.

6

127

1972

1973

1974

1975197619771978

1979

1980

1981

1982

19831984

1985

19861987

1988

1989

199019911992

1993

19941995199619971998

128

Table 25 – Incident Remediation Strategies

ID AMI Incident Classification

Remediation Strategy

REM.1 Theft of Energy The organization should determine a quantitative threshold of events that should occur before an AMI incident response team is deployed for investigation.

If theft of energy has been determined, the incident is reported to the business department of the AMI entity in order to recover lost revenue.

If the incident has been determined to be a part of a large scale incident the AMI entity may deploy its incident response team for further investigation.

The organization should determine a threshold (e.g., monetary) at which local, state, or federal law enforcement may be notified to assist in the investigation.

REM.2 Theft of C12.18 Logon Credentials

The organization should determine a quantitative threshold of failed C12.18 login attempts before an AMI incident response team is deployed for investigation. The organization should focus on events that occur outside of maintenance windows to reduce false positive detections.

If the C12.18 credentials have been determined to be compromised, the incident response team should assess the impact of the stolen credentials on the AMI logical architecture. Based on the impact assessment the incident response team may accept the risk or deploy a remediation strategy such as replacing the C12.18 login credentials on affected AMI meters.

REM.3 Bypass of Security Controls

The organization should determine a quantitative threshold of failed C12.22 authentication events before an AMI incident response team is deployed for investigation.

If security controls have been determined to have been bypassed, the incident response team should assess the impact of the bypass of security controls. The incident response team should then attempt to investigate the root cause of the event and may decide to deploy a remediation such as issuing a software patch.

REM.4 Failed Communications Integrity

The incident is reported to the AMI operations department of the AMI entity in order to correct the poor communications link.

If the incident has been determined to be a malicious attack the incident response team should work with the organization to determine the appropriate course of action.

REM.5 Power Outage The incident is reported to the AMI operations department of the AMI entity and organizational power restoration procedures are followed.

REM.6 Software Error The incident response team should work with the vendor to assess the impact of the software error on the AMI meter. The incident response team should then attempt to investigate the root cause of the event and may decide to deploy a remediation such as issuing a software patch.

In certain circumstances, an organization may encounter a new or unidentified vulnerability for which a remediation currently does not exist. In this case, the AMI entity should deploy an

7

129

1999

2000

200120022003

incident response team to investigate the incident and develop an incident response strategy and update the AMI incident response plan as necessary.

7.1.3 Incident ReviewOrganizations should periodically review their incident handling procedures following the occurrence of an AMI incident for effectiveness. The following sections provide guidance on the activities to be performed following an AMI incident.

7.1.3.1 Forensic Analysis

A forensics analysis is the process of collecting digital evidence such as AMI event logs, for the purposes of a legal investigation.

7.1.3.1.1 Guidance

In some circumstances the root cause of an AMI incident cannot be determined before the threat has been remediated. A forensic analysis of AMI event logs should be performed in order to determine the root-cause of the incident.

Depending on the nature of the incident and the AMI architecture, the organization should collect and archive logs from all components of the AMI logical architecture.

7.1.3.2 Incident Reporting [9 – SG.IR-11]

Depending on the nature of the event (e.g., widespread energy theft), the organization should report the incident to the appropriate authority if there are implications of activities violating local, state, or federal laws.

7.1.3.2.1 Guidance

The AMI entity should follow existing guidance for reporting incidents.

7.1.3.3 Incident Impact Analysis [9 – SG.IR-9]

Performing an incident impact analysis allows the organization to determine the actual impact of the incident in order to update their AMI incident response plans.

7.1.3.3.1 Guidance

The AMI entity should following existing guidance for performing an incident impact analysis.

8

130

200420052006

2007

200820092010

2011

20122013

2014

201520162017

20182019

2020

202120222023

2024

2025

2026

20272028

2029

2030

2031

131

8CONCLUSIONThe preceding sections highlight the key elements of AMI Incident Response in an AMI technology agnostic fashion. The information contained herein is the accumulation and combination of industrial input and experience. As such, it represents the current best practices for AMI incident response as AMI systems currently exist.

This document is a snapshot in time and must evolve as the AMI systems evolve. Future AMI operational experience and greater adoption of AMI security standards should be added to this document in the form of more elaborate and specific incident response guidelines.

C-1

132

2032

2033

2034203520362037

203820392040

133

Export Control Restrictions

Access to and use of EPRI Intellectual Property is granted with the specific understanding and requirement that responsibility for ensuring full compliance with all applicable U.S. and foreign export laws and regulations is being undertaken by you and your company. This includes an obligation to ensure that any individual receiving access hereunder who is not a U.S. citizen or permanent U.S. resident is permitted access under applicable U.S. and foreign export laws and regulations. In the event you are uncertain whether you or your company may lawfully obtain access to this EPRI Intellectual Property, you acknowledge that it is your obligation to consult with your company’s legal counsel to determine whether this access is lawful. Although EPRI may make available on a case-by-case basis an informal assessment of the applicable U.S. export classification for specific EPRI Intellectual Property, you and your company acknowledge that this assessment is solely for informational purposes and not for reliance purposes. You and your company acknowledge that it is still the obligation of you and your company to make your own assessment of the applicable U.S. export classification and ensure compliance accordingly. You and your company understand and acknowledge your obligations to make a prompt report to EPRI and the appropriate authorities regarding any access to or use of EPRI Intellectual Property hereunder that may be in violation of applicable U.S. or foreign export laws or regulations.

The Electric Power Research Institute Inc., (EPRI, www.epri.com) conducts research and development relating to the generation, delivery and use of electricity for the benefit of the public. An independent, nonprofit organization, EPRI brings together its scientists and engineers as well as experts from academia and industry to help address challenges in electricity, including reliability, efficiency, health, safety and the environment. EPRI also provides technology, policy and economic analyses to drive long-range research and development planning, and supports research in emerging technologies. EPRI’s members represent more than 90 percent of the electricity generated and delivered in the United States, and international participation extends to 40 countries. EPRI’s principal offices and laboratories are located in Palo Alto, Calif.; Charlotte, N.C.; Knoxville, Tenn.; and Lenox, Mass.

Together…Shaping the Future of Electricity

© 2010 Electric Power Research Institute (EPRI), Inc. All rights reserved. Electric Power Research Institute, EPRI, and TOGETHERSHAPING THE FUTURE OF ELECTRICITY are registered service marks of the Electric Power Research Institute, Inc.

10xxxxx

1

134

2041

Enter Report Title Here - UCAIugosgug.ucaiug.org/utilisec/Shared Documents/EPRI... · Web viewHow...

Documents

Transcript of Enter Report Title Here - UCAIugosgug.ucaiug.org/utilisec/Shared Documents/EPRI... · Web viewHow...