Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration...

22
1 Substation CIP Change Management Program SPP CIP Users Group Dawn Berndt June 2014

Transcript of Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration...

Page 1: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

1

Substation CIP Change Management Program

SPP CIP Users Group Dawn Berndt June 2014

Page 2: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

2

Agenda Background Substation CIP Change Management Program Challenges Lessons Learned

Page 3: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

3

Xcel Energy Background Serves 3.4 million electric customers in 8 states Three Operating Companies

Northern States Power (NSP) Public Service Company (PSCo) Southwestern Public Service (SPS)

NERC Compliance Regions MRO (NSP) WECC (PSCo) SPP (SPS)

Page 4: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

4

CIP Change Control Substations

78 CIP substations with 1,100 Critical Cyber Assets 16,000 work orders per year at CIP substations More than 100 change control forms processed

each year Opportunity for change control errors is large

Change Control Program continues to evolve

Page 5: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

5

CIP Change Control Key Players

CIP Consultant

Substation Field Engineering

Project Initiator

Engineering

Operations & Maintenance

Field Technician

Page 6: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

6

CIP Substation Change Control

Pro

ject

/Wor

k In

tiatit

edC

IP S

ubst

atio

n / C

yber

Ass

ets

7 D

AYS

23 D

AYS

CIP Consultant

Project InitiatorsSFE -Substation Field Engineering

SED – Substation Engineering DesignSCE – Substation Commissioning

EngineeringSPE – System Protection Engineering

CO&M

YesNo

Store Project Spreadsheet in

ProjectWise

CIP Sub <AND>Cyber Assets

Impacted?

Record all affected Cyber

Assets in Spreadsheet

Update CIP ESP Diagram

Done

CIP Project Spreadsheet

Change Control not needed,

inform InitiatorProcess Done

Update CIP Asset Inventory

Issue Project Spreadsheet

to field

PROJECTS Small Capital or Maintenance Large Capital Small Special O&M Install or Upgrades

Normal or Emergency Maintenance

YesCIP Project

SpreadsheetRecord device information on Spreadsheet

Perform required configuration and testing

activities

Return data to CIP

Consultant

Substation Work

Is Substation Work Complete?<OR>

Has remote connectivity been established for the first time?

Yes 7 DAYS Start

Complete Project Spreadsheet

documentation

No

CIP Substation Change ControlP

roje

ct/W

ork

Intia

tited

CIP Consultant

Project InitiatorsSFE -Substation Field Engineering

SED – Substation Engineering DesignSCE – Substation Commissioning

EngineeringSPE – System Protection Engineering

Substation O&M

YesNo

CIP Sub <AND> Cyber Assets

Impacted?

CIP Project Spreadsheet

PROJECTS Small Capital or Maintenance Large Capital Small Special O&M Install or Upgrades

Normal or Emergency Maintenance

YesCIP Project

SpreadsheetRecord device information on Spreadsheet

Perform required configuration and testing

activities

Page 7: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

7

CIP Substation Change Control

Pro

ject

/Wor

k In

tiatit

edC

IP S

ubst

atio

n / C

yber

Ass

ets

7 D

AYS

23 D

AYS

CIP Consultant

Project InitiatorsSFE -Substation Field Engineering

SED – Substation Engineering DesignSCE – Substation Commissioning

EngineeringSPE – System Protection Engineering

Substation O&M

YesNo

Store Project Spreadsheet in

ProjectWise

CIP Sub <AND> Cyber Assets

Impacted?

Record all affected Cyber

Assets in Spreadsheet

Update CIP ESP Diagram

Done

CIP Project Spreadsheet

Change Control not needed,

inform InitiatorProcess Done

Update CIP Asset Inventory

Issue Project Spreadsheet

to field

PROJECTS Small Capital or Maintenance Large Capital Small Special O&M Install or Upgrades

Normal or Emergency Maintenance

YesCIP Project

SpreadsheetRecord device information on Spreadsheet

Perform required configuration and testing

activities

Return data to CIP

Consultant

Substation Work

Is Substation Work Complete?<OR>

Has remote connectivity been established for the first time?

Yes 7 DAYS Start

Complete Project Spreadsheet

documentation

No

C

IP S

ubst

atio

n / C

yber

Ass

ets

YesNo

Record all affected Cyber

Assets in Spreadsheet

CIP Project Spreadsheet

Change Control not needed,

inform InitiatorProcess Done

Issue Project Spreadsheet

to field

YesCIP Project

SpreadsheetRecord device information on Spreadsheet

Perform required configuration and testing

activities

Substation Work

Is Substation Work Complete?<OR>

Has remote connectivity been established for the first time?

Yes 7 DAYS Start

No

Page 8: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

8

CIP Substation Change Control

Pro

ject

/Wor

k In

tiatit

edC

IP S

ubst

atio

n / C

yber

Ass

ets

7 D

AYS

23 D

AYS

CIP Consultant

Project InitiatorsSFE -Substation Field Engineering

SED – Substation Engineering DesignSCE – Substation Commissioning

EngineeringSPE – System Protection Engineering

Substation O&M

YesNo

Store Project Spreadsheet in

ProjectWise

CIP Sub <AND> Cyber Assets

Impacted?

Record all affected Cyber

Assets in Spreadsheet

Update CIP ESP Diagram

Done

CIP Project Spreadsheet

Change Control not needed,

inform InitiatorProcess Done

Update CIP Asset Inventory

Issue Project Spreadsheet

to field

PROJECTS Small Capital or Maintenance Large Capital Small Special O&M Install or Upgrades

Normal or Emergency Maintenance

YesCIP Project

SpreadsheetRecord device information on Spreadsheet

Perform required configuration and testing

activities

Return data to CIP

Consultant

Substation Work

Is Substation Work Complete?<OR>

Has remote connectivity been established for the first time?

Yes 7 DAYS Start

Complete Project Spreadsheet

documentation

No

C

7

DA

YS23

DA

YS

Store Project Spreadsheet in

ProjectWise

Update CIP ESP Diagram

Done

Update CIP Asset Inventory

Return data to CIP

Consultant

Yes 7 DAYS Start

Complete Project Spreadsheet

documentation

Page 9: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

9

Page 10: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

10

Page 11: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

11

Page 12: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

12

Page 13: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

13

Page 14: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

14

Page 15: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

15

Page 16: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

16

CIP Change Control Challenges

Manual change control process Rely on personnel to initiate change control for CIP substations Multiple departments required to fill out information

Depend on human performance for documentation Time Accuracy Completeness

Large number of people had access to CIP Substations Almost 25% were external parties (other utilities, contractors) Paring this list down

Training and awareness for all affected personnel on correct procedure

Page 17: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

17

CIP Change Control Lessons Learned

Workflow automation is crucial when dealing with many touch points in CIP Substations Identify process handoffs and consider controls

Importance of ongoing training and awareness Employee turnover Contractors/Consultants Face-to-face is valuable with field resources

Implementing compliance ‘controls’ can help prevent and detect issues

Page 18: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

18

CIP Change Control Controls Implemented

Established controls to help identify substation work requiring change control upfront Automated biweekly report to review new work orders Established process for checking CIP status on new substation

design projects Implemented manual security testing of access points Conduct bi-weekly change control meetings in each Operating

Company with Engineering, Operations and CIP Compliance groups Quarterly monitoring program Implemented tagging of disconnected device cables Installed access point “CIP notice” labels

Page 19: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

19

CIP Change Control Improved Field Awareness

Page 20: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

20

Port plugs and locks to prevent inadvertent connections

Technician guide for connecting and making changes to cyber assets in the Electronic Security Perimeter

Restructuring process documentation Change control awareness posters in CIP substations

CIP Change Control Controls In Development

Page 21: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

21

Implement additional controls Asset management solution (not Excel spreadsheets) Investigate automation solutions

Change control workflow Configuration management

Evaluate viability of additional testing facilities

CIP Change Control Preparation for CIP V5

Page 22: Substation CIP Change Management Program · CIP Change Control Lessons Learned ... Configuration management Evaluate viability of additional testing facilities CIP Change Control

22

Questions?


Cleaning-In-Place (CIP) Spray Nozzles Ed_2... · Cleaning-In-Place (CIP) Spray Nozzles ... sizes tank cleaning machine is recommended as it gives cyclic control speed of rotation

Cleaning-In-Place (CIP) Spray Nozzles Ed_2... · Cleaning-In-Place (CIP) Spray Nozzles ... sizes tank cleaning machine is recommended as it gives cyclic control speed of rotation

Standard CIP–002–4 - SCADAhacker - CIP v4.pdfStandard CIP–002–4 — Cyber Security — Critical Cyber Asset Identification ... Purpose: NERC Standards CIP-002-4 through CIP-009-4

Standard CIP–002–4 - SCADAhacker - CIP v4.pdfStandard CIP–002–4 — Cyber Security — Critical Cyber Asset Identification ... Purpose: NERC Standards CIP-002-4 through CIP-009-4

Bringing Climate Change into Official Plans: National (CIP ... · National (CIP) & Local (RPP) Perspective Clean Air Partnership September 21, 2018. Overview Introduction Canadian

Bringing Climate Change into Official Plans: National (CIP ... · National (CIP) & Local (RPP) Perspective Clean Air Partnership September 21, 2018. Overview Introduction Canadian

TRAFFIC CONTROL GUIDELINES - Harris County, …eng.hctx.net/cip/guidelines/TCP_Guidelines_and_Drawings.pdfTRAFFIC CONTROL GUIDELINES ... projects with spot or short speed reduction

TRAFFIC CONTROL GUIDELINES - Harris County, …eng.hctx.net/cip/guidelines/TCP_Guidelines_and_Drawings.pdfTRAFFIC CONTROL GUIDELINES ... projects with spot or short speed reduction

Inspection Report for WHO€¦ · CC Change Control CFU Colony-Forming Unit CIP Cleaning In Place CoA Certificate of Analysis CpK Process capability CQA Critical Quality Attribute

Inspection Report for WHO€¦ · CC Change Control CFU Colony-Forming Unit CIP Cleaning In Place CoA Certificate of Analysis CpK Process capability CQA Critical Quality Attribute

CIP-010-1 – Cyber Security – Configuration Change ...€¦ · Web viewCIP-010-1 – Cyber Security – Configuration Change Management & Vulnerability Assessments Reliability

CIP-010-1 – Cyber Security – Configuration Change ...€¦ · Web viewCIP-010-1 – Cyber Security – Configuration Change Management & Vulnerability Assessments Reliability

Audit Benefits of a ‘Closed-Loop Control Integration’ Between NERC CIP Vendors

Audit Benefits of a ‘Closed-Loop Control Integration’ Between NERC CIP Vendors

Brewery & CIP Automation Systems · MBAA District NW Fall Meeting October 8th & 9th, 2010 Automation Control Systems 2 ... " Automatic control ... " Distributed Control System (DCS)

Brewery & CIP Automation Systems · MBAA District NW Fall Meeting October 8th & 9th, 2010 Automation Control Systems 2 ... " Automatic control ... " Distributed Control System (DCS)

CIP 2019 Project Outline ~ Information Technology · 2018-12-21 · CIP 2019 Project Outline ~ Information Technology interaction with , etc. o Engagement - Change Champions/Rangers

CIP 2019 Project Outline ~ Information Technology · 2018-12-21 · CIP 2019 Project Outline ~ Information Technology interaction with , etc. o Engagement - Change Champions/Rangers

the cip report · Featured in this month’s issue of The CIP Report are Supervisory Control and Data Acquisition (SCADA) systems. SCADA systems monitor and control the processes

the cip report · Featured in this month’s issue of The CIP Report are Supervisory Control and Data Acquisition (SCADA) systems. SCADA systems monitor and control the processes

NERC CIP -1, -2 and -3: A Comparison - NERC CIP Version 6 ... · PDF fileNERC CIP Standards Comparison: -1 to -2 Standard v1 Req Number v1 Requirement Did the requirement change? v2

NERC CIP -1, -2 and -3: A Comparison - NERC CIP Version 6 ... · PDF fileNERC CIP Standards Comparison: -1 to -2 Standard v1 Req Number v1 Requirement Did the requirement change? v2

Change Control Management

Change Control Management

CIP FOR DAIRY INDUSTRY - JournalNXAs shown in fig.7 control panel of CIP system in this control panel we can control the system in the form of two switches and two indicator.when on

CIP FOR DAIRY INDUSTRY - JournalNXAs shown in fig.7 control panel of CIP system in this control panel we can control the system in the form of two switches and two indicator.when on

Sage FAS Fixed Assets - Back BayFAS_Fixed_Asset_Inventory]FAS-in… · FAS CIP Accounting Take control over your fixed assets even before they become fixed assets with FAS CIP Accounting.

Sage FAS Fixed Assets - Back BayFAS_Fixed_Asset_Inventory]FAS-in… · FAS CIP Accounting Take control over your fixed assets even before they become fixed assets with FAS CIP Accounting.

Inspection Report for WHO...CAPA Corrective Actions and Preventive Actions CC Change Control CFU Colony-Forming Unit CIP Cleaning In Place CoA Certificate of Analysis CpK …

Inspection Report for WHO...CAPA Corrective Actions and Preventive Actions CC Change Control CFU Colony-Forming Unit CIP Cleaning In Place CoA Certificate of Analysis CpK …

CIP Overview 1. Purposes of the CIP What Does CIP Success Look Like? Identifying the Team Roles of LEA, Principal, and CIP Team CIP Timeline 2.

CIP Overview 1. Purposes of the CIP What Does CIP Success Look Like? Identifying the Team Roles of LEA, Principal, and CIP Team CIP Timeline 2.

10 cip rome cip

10 cip rome cip

the cip reportcip.gmu.edu/wp-content/uploads/2013/06/79_The-CIP-Report... · 2018-04-26 · Featured in this month’s issue of The CIP Report are Supervisory Control and Data Acquisition

the cip reportcip.gmu.edu/wp-content/uploads/2013/06/79_The-CIP-Report... · 2018-04-26 · Featured in this month’s issue of The CIP Report are Supervisory Control and Data Acquisition

CIP-003-7 & CIP-003-8 Effective Dates and... · CIP-003-8 R4 Delegate CIP Senior Manager authority 7/1/2016 9. CIP Reliability Standard Effective Date CIP-005-6 July 1, 2020 CIP-008-6

CIP-003-7 & CIP-003-8 Effective Dates and... · CIP-003-8 R4 Delegate CIP Senior Manager authority 7/1/2016 9. CIP Reliability Standard Effective Date CIP-005-6 July 1, 2020 CIP-008-6

CIP-007/R1 System Configuration Baseline/Security Control Testing

CIP-007/R1 System Configuration Baseline/Security Control Testing