State of Reliability Report 2013

Jessica Bian, Director of Performance Analysis

Reliability Assessment and Performance Analysis (RAPA), NERC

Risk Issues Steering Committee Meeting, July 11-12, 2013

RELIABILITY | ACCOUNTABILITY 2

State of Reliability Report

• NERC’s annual, independent assessment of reliability performance

• Provides an integrated view of reliability performance

• The report assesses 18 metrics that address the characteristics of an adequate level of reliability

• Guidance that enables industry and policy makers to take effective steps and make sound decisions supporting reliability

• The Report is a step forward and it delivers analytical information which can be used for

Project Prioritization

Process Improvement

Characterization and Identification of Risk

Development of coordinated and multifunctional solutions

RELIABILITY | ACCOUNTABILITY 3

Metrics and Performance Analysis

• History

• 2008 - Reliability Metrics Working Group formed, reporting to NERC Operating Committee (OC) and Planning Committee (PC)

• 2009 – 9 reliability metrics approved by OC and PC, details at: http://www.nerc.com/pa/RAPA/PA/Performance%20Analysis%20DL/RMWG_Metric_Report-09-08-09.pdf

• 2010 – Another 9 metrics approved by OC and PC, first annual performance report published: http://www.nerc.com/pa/RAPA/PA/Performance%20Analysis%20DL/RMWG_AnnualReport6.1.pdf

• 2011 – Second reliability performance report published: http://www.nerc.com/pa/RAPA/PA/Performance%20Analysis%20DL/2011_RARPR_FINAL.pdf

RELIABILITY | ACCOUNTABILITY 4

Performance Analysis

• History (continued)

• 2012 – First annual state of reliability report

• 2013 – Second state of reliability report

• New in 2013 – Advanced risk identification methods Consistent and predicable pattern

Correlation between initiating factors and transmission outage events

Used in other industries (NASA, NRC, etc.) – extremely effective

RELIABILITY | ACCOUNTABILITY 5

2013 State of Reliability Report

• Prepared by NERC with guidance from the Performance Analysis Subcommittee (PAS)

• Significant Contributions from: Operating Committee (OC) o Resources Subcommittee (RS)

o Frequency Working Group (FWG)

o Event Analysis Subcommittee (EAS)

o Operating Reliability Subcommittee (ORS)

Planning Committee (PC) o Reliability Assessment Subcommittee (RAS)

o System Protection and Control Subcommittee (SPCS)

o Transmission Availability Data System Working Group (TADSWG)

o Generating Availability Data System Working Group (GADSWG)

o Demand Response Availability Data System Working Group (DADSWG)

Compliance and Certification Committee (CCC)

RELIABILITY | ACCOUNTABILITY 6

BPS Reliability Remains Adequate

• Bulk power system reliability stable within ALR conditions For 2008 to 2012, no significant upward or downward trends

Consistent 97% AC circuit availability

12 14

11 12

8 6

14

9 9 9

2 0

2

4

6

8

10

12

14

16

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Even

ts

Year

Bulk Power System Transmission Related Events Resulting in Load Loss

RELIABILITY | ACCOUNTABILITY 7

BPS Reliability Remains Adequate

NERC Annual Daily Severity Risk Index (SRI) Sorted Descending with Historic Benchmark Days

RELIABILITY | ACCOUNTABILITY 8

Highest 8 Stress Days (2008-2012)

NERC Annual Daily Severity Risk Index (SRI) Sorted Descending with Historic Benchmark Days

RELIABILITY | ACCOUNTABILITY 9

Key Compliance Monitoring Indicator Trends Improving

NERC Confirmed Violations by Assessed Risk (June 18, 2007 – December 31, 2012)

Assessed

Risk BA

L

CIP

CO

M

EOP

FAC

INT

IRO

MO

D

NU

C

PER

PR

C

TOP

TPL

VA

R

ALL

Minimal 80 1806 56 242 515 26 64 34 1 99 853 193 125 268 4362 85.3%

Moderate 2 410 13 27 24 1 4 11 116 34 3 27 672 13.1%

Serious 31 2 3 6 2 3 23 7 4 81 1.6%

Total 82 2247 71 272 544 27 71 34 1 113 992 234 128 299 5115

50

55

60

65

70

75

80

85

90

95

100

KC

MI (

%)

Time Period

Key Compliance Monitoring Index Trend (2008-2012)

RELIABILITY | ACCOUNTABILITY 10

Frequency Response (FR) Stable

• Consistent data collection/procedures implemented in 2011

• Further analysis for EI FR events with less than 1,500 MW/0.1Hz

Eastern Interconnection

ERCOT Interconnection

44

51

49

67 65

70

(MW

/0.1

Hz)

RELIABILITY | ACCOUNTABILITY 11

Misoperations Significant Risk Issue

• Protection System Misoperations are a Significant Contributor to Disturbance Events and Automatic Transmission Outage Severity Largest positive correlation with 2012 automatic transmission outage severity

Statistically significant

Highest relative risk

• Industry Actionable Steps Recommended 65% from three leading causes

Summarized in Protection System Misoperations Task Force (PSMTF) report

Performance Measure Implemented

RELIABILITY | ACCOUNTABILITY 12

Misoperations Trend

27

7

27

2 30

6

24

7

22

9

14

8

15

2

14

7

89

90

12

5

14

3

16

1 2

14

20

1 2

54

18

7

14

1

16

5

10

1 13

1

0

50

100

150

200

250

300

350

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Mis

op

era

tio

ns

Co

un

t

Month

Misoperations by Month (2011Q2-2012Q4)

2011 2012

RELIABILITY | ACCOUNTABILITY 13

Misoperation Rate by Regional Entity (2012Q4)

11.7%

9.6%

16.8%

9.7%

13.9%

7.2%

0

2%

4%

6%

8%

10%

12%

14%

16%

18%

FRCC MRO NPCC* RFC SERC SPP TRE WECC*

Mis

op

erat

ion

Rat

e (#

Mis

op

era

tio

ns/

# O

pe

rati

on

s)

Regional Entity

Misoperation Rate by Regional Entity in 2012Q4

*NPCC and WECC Misoperation Rate Not Included.

RELIABILITY | ACCOUNTABILITY 14

• Reviewed historical (pre-2011) Regional Entity misoperations

• Analyzed 1,500+ misoperations from January 1, 2011 to April 1, 2012 submitted in NERC misoperations template

• Focused on top 3 misoperation causes: Incorrect settings/logic/design

Relay failure/malfunction

Communication failures

• Classified misoperations in top 3 misoperation causes by sub causes

• Based on the analysis of top issues, developed suggestions for improvement to reduce top misoperation areas

• Recommended data monitoring and data improvements

Protection System Misoperations Task Force (PSMTF) Analysis

RELIABILITY | ACCOUNTABILITY 15

Misoperations by Cause Code

Incorrect setting / logic / design errors, 628,

28%

Relay failures / malfunctions, 465, 20%

Communication failures, 396, 17%

AC system, 193, 8%

As-left personnel

error, 201, 9% DC system, 112, 5%

Unknown / unexplainable, 273,

12%

Other, 11, 1%

NERC-wide Misoperations by Cause Code (Jan 1, 2011 – April 1, 2012)

RELIABILITY | ACCOUNTABILITY 16

• Proper Application of Relay Elements

• Settings Errors

• Microprocessor-based Relay Firmware

• Application of Power Line Carrier Communication Aided Protection

• Performance Monitoring and Data Improvements

Suggestions for Improvement

RELIABILITY | ACCOUNTABILITY 17

• Misoperations due to setting errors can potentially be reduced.

• Techniques that could be used to reduce the application of incorrect settings include: Peer reviews

Increased training

More extensive fault studies

Standard templates for setting standard schemes using complex relays

Periodic review of existing settings when there is a change in system topography

• The IEEE Power System Relaying Subcommittee, IEEE PSRC, published a working group report to provide additional technical guidance for protective relay setting quality control. http://www.pes-psrc.org/Reports/Processes_Issues_Trends_and_Quality_Control_of_Relay_Settings.pdf

Settings Errors

RELIABILITY | ACCOUNTABILITY 18

Risk Patterns/Event Analysis

• NERC Event Analysis 2012

o 116 Category 1-5 Events

o 42 Events (36%) with misoperations

• ALR1-4 - Transmission outage events led to firm load loss Only Two events in 2012

o Oct 15, 2012, 665 MW firm load loss due to relay misoperations

o Dec 6, 2012, 300 MW+ firm load loss due to SPS/RAS misoperation

• Preliminary 2013 Event Analysis as of 4/30/2013 33 events, 10 with misoperations

RELIABILITY | ACCOUNTABILITY 19

• One of High Priority Risk Issues from RISC

• 15+ Lessons Learned distributed since 2010

• PSMTF Report – Approved by PC in April 8, 2013 Developed targeted, actionable solutions for registered entities to reduce

misoperations

Enhance data reporting, including subcause codes

• NERC Industry Webinar – May 7, 2013 Highlighted three identified primary causes

Suggested registered entities to apply the solutions based on their particular circumstances.

NERC/Industry Actions

RELIABILITY | ACCOUNTABILITY 20

• Training Modules for Each Primary Cause – Target to complete by July 2013 Provide further assistance to the entities that need guidance the most

Share industry example of excellences

• On-Going Performance Monitoring (ALR4-1) PC directed its Protection System and Control Subcommittee (SPCS) at the

June 11-12, 2013 meeting

o Enhance reporting accuracy, including subcause codes for the three primary causes

o Reduce percentage of misoperation reporting that does not have sufficient information

Misoperation rate data became available first time for the 2012Q4 reporting period

NERC Protection System Misoperations Dashboard

http://www.nerc.com/pa/RAPA/ri/Pages/ProtectionSystemMisoperations.aspx

NERC/Industry Actions

RELIABILITY | ACCOUNTABILITY 21

AC Substation Equipment Failure Second Significant Risk Issue

• AC Substation Equipment Failures are a Second Significant Contributor to Disturbance Events and Automatic Transmission Outage Severity Second largest positive correlation with 2012 automatic transmission outage

severity

Statistically significant

Second highest relative risk

• Recommendations Form a technical group

Investigate root causes

Performance measure implemented

RELIABILITY | ACCOUNTABILITY 22

• PC approved the formation of the AC Substation Equipment Task Force at the June 11-12, 2013 meeting Conduct a thorough investigation into the root causes of failed AC

substation equipment failure that contribute to disturbance events

Provide risk control solutions to improve performance

Final report by June 2014

o Endorsement from OC

o Approval from PC

• Collaborate with EAS/TWG and PAS/TADSWG Avoid duplication regarding related work by the TWGS and TADSWG

NERC/Industry Actions

RELIABILITY | ACCOUNTABILITY 23

Risk Pattern/Event Analysis

• NERC Event Analysis

2011Q2 – 2012Q3Q02

o 121 Category 1-5 Events

o 25 Events (21%) with AC substation equipment failures

26 AC Substation Equipment Failures in 25 Events

Circuit Breaker

77%

Reactor 4%

Transformer 15%

Disconnect Switch

4%

RELIABILITY | ACCOUNTABILITY 24

Transmission Events with Unknown Cause Warrant Analysis

• 19% automatic transmission outage events with Unknown initiating cause 10% common/dependent mode outages

34% sustained outages with Unknown

• Recommendations Form a technical group

Enhance reporting clarifications and consistency

• PC directed TADSWG to address the issue at the June 11-12, 2013 meeting

RELIABILITY | ACCOUNTABILITY 25

Q/A

Questions and Answers

Additional background material is located at: State of Reliability Report 2013: http://www.nerc.com/pa/RAPA/PA/Performance%20Analysis%20DL/2013_SOR_May%2015.pdf NERC Reliability Indicators: http://www.nerc.com/pa/RAPA/Pages/ReliabilityIndicators.aspx Approved Metrics: http://www.nerc.com/comm/PC/Pages/Performance%20Analysis%20Subcommittee%20(PAS)/Approved-Metrics.aspx

Contacts

Jessica Bian, 404-446-9702 Jessica.Bian@nerc.net Naved Khan, 404-446-9730 Naved.Khan@nerc.net

Background Materials

- NERC 18 Reliability Performance Metrics

RELIABILITY | ACCOUNTABILITY 27

Objective

• Provide a high-level overview of the 2008-2012 reliability metrics and trends Analysis of reliability performance and trends will help develop

quantitative risk control measures to better prioritize and inform reliability management business decisions.

RELIABILITY | ACCOUNTABILITY 28

• NERC Planning Committee (PC) endorsed the formation of Reliability Metric Working Group at its December 2007 meeting Now known as Performance Analysis Subcommittee

• A slate of metrics which can yield an overall assessment of the North American Bulk Power System reliability

• Identify and track key reliability indicators as a means of benchmarking reliability performance

• Within the context of the “Adequate Level of Reliability” (ALR) framework Adequate Level of Reliability” describes the fundamental concepts and

six characteristics needed to achieve an adequate level of reliability

• Identify indication of potential reliability concerns

Background

RELIABILITY | ACCOUNTABILITY 29

• Number of Metrics Approved - 18

Metrics

ALR* Characteristics

Boundary

ALR1-5

ALR1-12

Contingencies

ALR1-4

ALR2-4

ALR2-5

Integrity

ALR3-5

Protection

ALR4-1

ALR2-3

Restoration

ALR5-1

Adequacy

ALR1-3

ALR6-1

ALR6-11

ALR6-12

ALR6-13

ALR6-14

ALR6-15

ALR6-16

ALR6-2

ALR6-3

* PAS is in the process of realigning current metrics with new ALR definition

RELIABILITY | ACCOUNTABILITY 30

Metrics Development TADS

Event Reports

ES&D OE-417

RCIS

DATA reporting Template

Data Sources

RELIABILITY | ACCOUNTABILITY 31

Metric Trend Rating

ALR Boundary Trend Rating

1-5 System Voltage Performance ** 1-12 Interconnection Frequency Response

Contingencies

1-4 BPS Transmission Related Events Resulting in Loss of Load

2-4 Average Percent Non-Recovery Disturbance Control Standard Events

2-5 Disturbance Control Events Greater than Most Severe Single Contingency *

Integrity

3-5 Interconnected Reliability Operating Limit/ System Operating Limit (IROL/SOL) Exceedances * Protection

2-3 Activation of Underfrequency Load Shedding

4-1 Automatic Transmission Outages Caused by Failed Protection System Equipment ** Adequacy

1-3 Planning Reserve Margin * 6-1 Transmission Constraint Mitigation

6-2 Energy Emergency Alert 3 (EEA3)

6-3 Energy Emergency Alert 2 (EEA2)

6-11 Automatic AC Transmission Outages Initiated by Failed Protection System Equipment

6-12 Automatic AC Transmission Outages Initiated by Human Error

6-13 Automatic AC Transmission Outages Initiated by Failed AC Substation Equipment

6-14 Automatic AC Transmission Outages Initiated by Failed AC Circuit Equipment

6-15 Element Availability Percentage

6-16 Transmission System Unavailability

Trend Rating Symbols Significant Improvement

Slight Improvement

No Change

Inconclusive/Mixed * Slight Deterioration

Significant Deterioration

New Data **

RELIABILITY | ACCOUNTABILITY 32

Improving Trend

ALR Title

1-4 BPS Transmission Related Events Resulting in Loss of Load

2-4 Average Percent Non-Recovery Disturbance Control Standard Events

RELIABILITY | ACCOUNTABILITY 33

• Positive Trend Indication

ALR1-4

7,085

64,850

4,950 8,942

3,763 2,249

11,045

4,432 4,078

10,212

1,055

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Tota

l An

nu

al L

oad

Lo

ss (

MW

)

Year

ALR1‐4 BPS Transmission-Related Events Resulting in Load Loss (2002–2012)

Colors represent individual events.

12 14

11 12

8 6

14

9 9 9

2 0

5

10

15

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Even

ts

Year

Bulk Power System Transmission Related Events Resulting in Load Loss

RELIABILITY | ACCOUNTABILITY 34

ALR2-4

FRCC

NPCC

SERC TRE

0 0.5

1 1.5

2

2.5

3

3.5

4

Pe

rce

nta

ge

Quarter

ALR2-4 : Average Percent Non-Recovery of DCS Events (2007 - 2012)

FRCC

MRO

NPCC

RFC

SERC

SPP

TRE

WECC

• Positive Trend Indication

RELIABILITY | ACCOUNTABILITY 35

No Clear Trends/Mixed Trend

ALR Title

2-5 Disturbance Control Events Greater than Most Severe Single Contingency

3-5 Interconnected Reliability Operating Limit/ System Operating Limit (IROL/SOL) Exceedances

1-3

Planning Reserve Margin

RELIABILITY | ACCOUNTABILITY 36

• Not Clear/Mixed

ALR2-5

FRCC

NPCC

SERC

TRE

0

1

2

3

4

5

6

7

Co

un

t

Quarter

ALR2-5 : Disturbance Control Events Greater Than Most Severe Single Contingency (2007-2012)

FRCC

MRO

NPCC

RFC

SERC

SPP

TRE

WECC

RELIABILITY | ACCOUNTABILITY 37

ALR3-5 IROL/SOL Exceedance

• Not Clear/Mixed

RELIABILITY | ACCOUNTABILITY 38

ALR1-3: Planning Reserve Margin

RELIABILITY | ACCOUNTABILITY 39

No Change

ALR Title

1-12 Interconnection Frequency Response

2-3 Activation of Under frequency Load Shedding

6-1 Transmission Constraint Mitigation

6-2 Energy Emergency Alert 3 (EEA3)

6-3 Energy Emergency Alert 2 (EEA2)

6-11 Automatic AC Transmission Outages Initiated by Failed Protection System Equipment

6-12 Automatic AC Transmission Outages Initiated by Human Error

6-13 Automatic AC Transmission Outages Initiated by Failed AC Substation Equipment

6-14 Automatic AC Transmission Outages Initiated by Failed AC Circuit Equipment

6-15 Element Availability Percentage

6-16 Transmission System Unavailability

RELIABILITY | ACCOUNTABILITY 40

No Data/New Data

ALR Title

1-5 System Voltage Performance

4-1 Automatic Transmission Outages Caused by Failed Protection System Equipment

RELIABILITY | ACCOUNTABILITY 41

NERC Metrics Dashboard

http://www.nerc.com/pa/RAPA/Pages/ReliabilityIndicators.aspx