2011 ReliabilityFirst 693 Compliance Audit Process for 6 Year Audit Cycle Entities Glenn Kaht Senior...
-
Upload
charles-murphy -
Category
Documents
-
view
219 -
download
1
description
Transcript of 2011 ReliabilityFirst 693 Compliance Audit Process for 6 Year Audit Cycle Entities Glenn Kaht Senior...
2011 ReliabilityFirst 693 Compliance Audit Process for 6 Year Audit
Cycle Entities
Glenn Kaht
Senior Consultant - Compliance ReliabilityFirst Corporation
January, 2011
05/03/23 2
Presentation Goals
The goals of this presentation are to: Discuss Compliance Audit references and define
“Compliance Audit” Discuss the Reliability Standards that are or may be
within the scope of a 2011 Compliance Audit and the audit review period
Provide an overview of the audit process for entities that are on a 6 year audit cycle
Answer questions regarding the 2011 Compliance Audit process for registered entities that are on a 6 year audit cycle
05/03/23 3
Audit Process References
Some references used in the performance of Compliance Audits: ReliabilityFirst Compliance Monitoring and
Enforcement Program (CMEP) NERC Rules of Procedure 2011 NERC and ReliabilityFirst Implementation Plans NERC 2011 Actively Monitored Reliability Standards ReliabilityFirst 2011 Compliance Monitoring Schedule Questionnaire-Reliability Standard Audit Worksheets
(Q-RSAWs)
05/03/23 4
Compliance Audit - Definition
What is a Compliance Audit?
“A systematic, objective review and examination of records and activities to determine whether a Registered Entity meets the requirements of applicable Reliability Standards.”
05/03/23 5
6 Year Audit Cycle Basis
NERC Rules of Procedure section 403.11.1: “For an entity registered as a balancing authority, reliability coordinator, or transmission operator, the compliance audit will be performed at least once every three years. For other bulk power system owners, operators, and users on the NERC Compliance Registry, compliance audits shall be performed on a schedule established by NERC.”
At this time, there are no plans to audit PSEs in 2011 Compliance Audits for other entities are to be performed at least
once every six years.
Compliance Audits of registered entities subject to a compliance audit at least once every six years will be conducted off-site from the facilities of the audited entity (although ReliabilityFirst may conduct audit activities on-site if deemed necessary).
05/03/23 6
Reliability Standards Within Audit Scope
Which Reliability Standards are within the scope of 2011 Compliance Audits? All applicable NERC Reliability Standards/requirements
identified to be monitored via Audit in the NERC 2011 Actively Monitored Reliability Standards list (unless NERC approves the exclusion)
Additional NERC Reliability Standards/requirements selected by ReliabilityFirst to be included within the scope of the Compliance Audit
ReliabilityFirst Standards approved by NERC and FERC
Open and completed mitigation plans will be reviewed by the audit team
05/03/23 7
Audit Review Period
In general, the audit review period for 2011 Compliance Audits is as follows: Current and 3 previous calendar years through the end of the
audit (i.e., January 1, 2008 through the end of the audit) Caveats: The start of the audit review period for a particular function will
be no earlier than the date that an entity is placed on the NERC compliance registry for that particular function.
If an entity was subject to a compliance audit within the 3 previous years, then the start of the 2011 audit review period corresponds to the end of the previous audit.
05/03/23 8
Data Retention Requirements
Data retention requirements for 2011 Compliance Audits Reference NERC Compliance Process Bulletin #2009-005 (Current In-Force
Document Data Retention Requirements for Registered Entities) issued on June 29, 2009
Generally consistent with data retention requirements identified within a particular Reliability Standard
Data retention section of PRC-005 specifies: “…shall retain evidence of the implementation of its Protection System maintenance and testing program for three years.”
Since the registered entity may specify a M&T interval longer than three years, the registered entity is expected to provide evidence of implementation of its Protection System M&T program for the entire review period
2011 Audit Process Overview
High level overview of 2011 Compliance Audits: 90 day audit notification to entity 85 day conference call with entity Entity submittal of pre-audit survey and sampling data
from Attachment C 30 days after receipt of 90 day notification
Entity submittal of completed Q-RSAWs and evidence 40 days before scheduled start date of audit
Audit team pre-audit review of evidence Off-Site Reviews Audit Report Completion
05/03/23 9
05/03/23 10
90 Day Audit Notification
A Compliance Audit Notification will be sent approximately 90 days prior to the start of the audit. The notification will include: 90 Day Audit Notification General Instructions Work history and participant agreements of
ReliabilityFirst audit team members Pre-audit survey Attachment A - List of Standards/Requirements within
the initial scope of the audit
05/03/23 11
90 Day Audit Notification – Cont’d
Attachment B - Entity Certification Signature form Attachment C – Data Sampling Evidence Spreadsheets Q-RSAWs for the NERC Standards within the initial
scope of the audit
90 Day Audit Notification and General Instructions
The 90 Day Audit Notification and General Instructions provide information and instructions regarding the audit and audit process and discusses information contained in the 90 day notification package (pre-audit survey, Q-RSAWs, etc.)
05/03/23 12
05/03/23 13
Work Histories and Participation Agreements
Work histories and participation agreements (e.g., Code of Business Conduct and Ethics, Confidentiality/Non-Disclosure) of the ReliabilityFirst audit team are provided to the audited entity.
Section 1500 of the NERC Rules of Procedure governs NERC staff (and the ReliabilityFirst audit team) responsibilities and obligations regarding Confidentiality.
Members of the audit team will not sign an entity specific confidentiality agreement.
05/03/23 14
Audit Team Makeup
The audit team will typically consist of 2 or more members with experience in Planning and/or Operations.
Audit Team Lead (Typically a member of the ReliabilityFirst Compliance Staff)
Audit Team Co-lead (if the audit team has 2 or more sub-teams)
Other team members or observers NERC observers and/or participants (@ NERC’s
discretion) FERC observers and/or participants (@ FERC’s
discretion)
05/03/23 15
Objection to a Team Member
A Registered Entity can object to an audit team member’s participation on the audit team: Objection may be based on the grounds of conflict of
interest, or the existence of other circumstances that could interfere with the team’s impartial performance of their duties
Objection must be provided in writing to ReliabilityFirst no later than 15 days prior to the start of the audit
ReliabilityFirst will make the final determination regarding the team member’s participation in the audit
NERC and FERC staff cannot be limited in their participation on an audit
05/03/23 16
Compliance Pre-Audit Survey
The pre-audit survey must be completed by the audited entity in order to provide the audit team:
General information of the organization, including contact information, registration details, organization profile, neighboring entities, etc.
Information regarding the audited entity’s internal compliance program and culture
Attachment A
Attachment A is a worksheet that:
Identifies all Standards/Requirements that are within the initial scope of the audit.
Identifies the applicable function(s) for each Requirement within the initial scope of the audit.
Can be used by the audit team and the audited entity to manage/track the audit scope and progress.
The audited entity should provide responses/evidence for each entry in Attachment A.
The scope of the audit may be expanded beyond the initial scope of the audit identified in Attachment A!
05/03/23 17
05/03/23 18
Attachment B
Attachment B - Entity Certification Signature form is to be completed and signed by an individual authorized to execute the Certification.
The individual who signs Attachment B is attesting that
the statements and supporting documents included in the response and appended to the certification are true and correct as of the date of signing.
The completed and signed Attachment B should be submitted to ReliabilityFirst at the same time that the evidence and completed Q-RSAWs are submitted.
Attachment C
In early 2011, the 90 day audit notification will include Attachment C – Data Sampling. Attachment C will include evidence/information requests for specific requirements.
Examples of items that may be requested:
Operators logs, voice recordings, etc. for specific days Evidence of submittal of study information for specific days List of entity equipment (substations, transmission and
generation protective equipment, UFLS relays, SPS equipment, etc.)
05/03/23 19
Attachment C – Cont’d
Attachment C is a tool that will be used by the audit team and the audited entity to compile certain evidence.
The use of Attachment C is intended to make the audit process more systematic and increase audit efficiency
Attachment C is not an all-inclusive listing of evidence that will need to be provided by the audited entity
05/03/23 20
Evidence Spreadsheet
In early 2011, the 90 day audit notification will include an Evidence Spreadsheet. The Evidence Spreadsheet: Is a guidance tool to be used by audited entities in their
compilation of evidence. Using the Evidence Spreadsheet does not ensure compliance but assists the entity and may increase efficiency for the audited entity and the audit team.
Is a listing of Standards/Requirements and types of evidence (agreements, procedures, logs, voice recording, etc.) that the entity should submit as evidence as per the requirements
Is not an all inclusive listing
The audit team may request additional substantiating evidence to assist the audit team in a determination of compliance
05/03/23 21
05/03/23 22
Q-RSAWs
Q-RSAWs: Audit worksheets for the Reliability Standards Provide guidelines concerning the
requirements (Compliance Assessment Approach)
Do not add additional requirements Posted on NERC Website Entity sections of the Q-RSAWs must be fully
completed and returned (including supporting evidence) 40 days before the scheduled start date of the audit
85 Day Conference Call
Approximately 85 days prior to the start of the audit, the Audit Team Lead will contact the audited entity to discuss the audit. Topics may include:
The 90 day notification package The pre-audit survey The Q-RSAWs Particular details of the audited entity Guidance on evidence submittalsAdditional questions from the audited entity
05/03/23 23
30 Day Submittals
No later than 30 days after receipt of the 90 day notification, the audited entity is to submit the following to ReliabilityFirst:
The completed pre-audit survey
Sampling evidence/information as specified in Attachment C
05/03/23 24
40 Day Submittal of Evidence
No later than 40 days prior to the scheduled start date of the audit, entities are to submit:
Completed Q-RSAWs Evidence of compliance to the
Standards/Requirements within the initial scope of the audit (Attachment A)
Completed and signed Attachment B
05/03/23 25
Audit Team Pre-Audit Reviews
After the initial evidence has been submitted, and prior to the scheduled start date of the audit, the audit team may conduct pre-audit reviews in order to: Schedule the opening presentation Review/discuss the evidence and information
submitted Make preliminary compliance determinations Develop additional requests for evidence as
necessaryMay be sent to the entity prior to the audit
05/03/23 26
05/03/23 27
Off-Site Reviews
The off-site reviews are conducted at the ReliabilityFirst offices and are expected to be completed within the assigned audit period that has been scheduled, but may be extended if necessary. It is not expected that the audited entity be present at the ReliabilityFirst offices during the reviews. The off-site reviews include: An opening presentation conducted by the audit team A review of compliance to the Standards/Requirements within
the scope of the audit An exit presentation scheduled and conducted by the audit team
05/03/23 28
Audit Team Opening Presentation
The audit team will conduct an opening presentation which will: Introduce the audit team members Review the authority of ReliabilityFirst Review the objectives and scope of the audit Discuss confidentiality issues Provide an overview of the audit process Discuss the role of SMEs Discuss evidence and types of evidence Answer questions related to the audit process
Off-site Reviews
The audit team will complete reviews of the evidence submitted by the audited entity. The audit team may request the audited entity to provide clarification of submitted evidence.
The audited entity should have SMEs available during the scheduled audit period.
Additional evidence may be requested by the audit team.
05/03/23 29
On-Site Visits
On-Site visits to entity facilities may be conducted as deemed necessary by the audit team.
05/03/23 30
05/03/23 31
Exit Presentation
At the conclusion of the audit, the audit team will conduct an exit presentation to: Review the audit scope Discuss the terms used in the audit findings Present the preliminary findings of the audit team
Explain the basis of any possible violations identified Review possible outcomes/actions resulting from possible
violations identified by the audit team (dismissal, notifications, appeals, settlement negotiations, mitigation plans, etc)
Discuss “Areas of Concern” identified by the audit team Discuss “Items for Consideration” identified by the audit team Discuss the audit report process and timeline Discuss feedback that the audited entity may provide regarding
the audit team and the audit process
05/03/23 32
Audit Report Completion
After the completion of the Compliance Audit, the audit team will develop a Compliance Audit report. There are 2 versions of the Compliance Audit report: Non-public version Public version (confidential information is redacted)
The audited entity will be provided the opportunity to review and comment on the audit report.
The Audit Team Lead develops a
draft report
The Audit Team Lead receives
comments from the Audit team
Audit Team provides
comments
The Audit Team Lead transmits the report for audit team
review
20 Business days
The Audit Team conducts an exit briefing
with the Registered Entity with preliminary
findings
Audit Team Lead sends the draft
report to the Audit Team for their
review and comments
The Audit Team Lead sends the
draft report to the Registered Entity for their review and comments
Audit Team Lead revises the draft
compliance report
The draft report is edited upon receipt of
Registered Entity
comments
Audit Team Lead
revises the report upon receipt of
Audit Team’s
comments
Send final report to RFC VP and
Director of Compliance, NERC and
Registered Entity
Audit Report Process and Timeline
20 business days
10 business days
5 business days
5 business days
Registered Entity reviews and
provide comments
Revision of the draft report
Audit Team provides
comments
5 business days
Audit Team Lead
completes final
compliance report
5 business days
2011 Compliance Audit Process 6 Year Audit Cycle
Questions?
34