Compliance Monitoring and Enforcement Audit Program Background Information

Introduction

This presentation is intended as an introduction to the audit process for employees of entities being audited by MACD. Please refer to the manual for more detail

2

Why is MACD Performing Compliance Audits?

• Electricity is critical to our society• The industry has established standards to

ensure reliable operation of the power system

• Under the Electricity Act (Section 32.1) , it is the IESOs role to monitor and enforce compliance with these standards

• IESO delegates these responsibilities to MACD

3

Reliability Standards

• Reliability Standards are created to improve the reliable operation of the bulk electricity system

• Minimum standards for operation in key areas

• Market Participants must continuously follow all applicable Reliability Standards in effect in Ontario

• Depending on your company role, different standards may apply e.g. Generator Operator Standards compared to Transmission Operator standards

4

Reliability Standards

• NERC – North American Electric Reliability Corporation establishes reliability standards. http://www.nerc.com/page.php?cid=2%7C20

• NPCC –North East Power Coordinating Council has delegated authority from NERC.

• Ontario is part of the NPCC, IESO participates in the development and approval of standard. Stakeholder involvement through the Reliability Standards Sub Committee of the IESO

• The Ontario Energy Board has the right to reject NERC reliability standards

5

Introduction to Compliance Audits

• Audits must provide reasonable assurance that evidence is sufficient and appropriate to support the auditors findings and conclusions.– Sufficient quantity of evidence, spans the time

period being audited– Evidence must be of sufficient quality –

authenticated– Typically involves multiple types of evidence

related to the same thing e.g.• Approved internal documentation• Physical examination/inspection• Analytical procedures with adequate data• Observations and inquiries

6

MACD Audit Program

• A ‘risk based’ audit program, not a pre set audit cycle

• MACD considers industry trends, Ontario specific inputs when developing the program

• MACD makes an assessment of the risk to reliability for various standards and participants

• The Audit Program begins with an implementation plan issued in the fall

7

MACD Implementation Plan

• Issued in the fall – after NERC issues their plan

• Provides a list of reliability standards which MACD will focus on during the upcoming year

• The implementation plan may be revised during the year depending on industry developments

• Individual audits will include standards from the implementation plan but may also include additional standards.

• Individual audits are initiated with an Audit Notification, are not listed in the Implementation Plan

8

Introduction to Compliance Audits

• Part Two of this presentation will deal with the audit process once an Audit Notification has been sent to a Market Participant. – Audit Notification– Pre Audit Meetings– Data Submission– The Audit– Exit Meeting– Audit Report

9

Compliance Monitoring and Enforcement Audit Program -

Background Information

End Of Presentation

10

Compliance Monitoring and Enforcement Audit Program - The Audit Process

Introduction

This presentation is intended as an introduction to the audit process for employees of entities being audited by MACD. Please refer to the manual for more detail

12

Introduction to Compliance Audits

• Audit Notification• Pre Audit Meetings• Data Submission• The Audit• Exit Meeting• Audit Report

13

Introduction to Compliance Audits

• Audits are not “out to get you”• “Trust but Verify” is the philosophy for the

process• The Audit team must, in their professional

judgment, be satisfied that the evidence demonstrates compliance

• If the Audit team cannot confirm compliance to a standard, the report will indicate potential non compliance which MAY trigger an investigation

14

Key Roles

Your company’s Audit Coordinator• The Single Point of Contact for the audit• Will do all coordination with MACD• The person you should talk to with any

concerns or questions• The only person who should identify

concerns or issues to the MACD Audit Team

• The person responsible for all logistics with the MACD Audit Team

15

Key Roles

Subject Matter Expert• The person your company has identified

as being able to discuss audit information with the audit team

• Essential to your company’s ability to demonstrate compliance with standards

• May be interviewed by the Audit Team to discuss reliability standards

• Should discuss any questions about the audit in advance with your Audit Coordinator 16

Key Roles

Audit Team Lead• An employee of MACD• Coordinates all audit efforts for MACD

with your Audit Coordinator• Discusses any issues or concerns with

your Audit Coordinator• The local decision maker for MACD

17

Audit Notification

• At least 90 days before your audit, you will receive a notification– Audit Team– Scope– Off Site vs. On Site– Date of Audit– Pre Audit Survey– Initial Data Requests– Pre Audit Schedule of Activities

18

Notification - Audit Team

• Audit Team Members:– are responsible for producing an

opinion on adherence to standards– have experience in the industry and/or

auditing– have completed NERC Audit Training– have confidentiality agreements in

place– are independent, have not worked in

your company in the recent past– issues? Discuss with MACD in advance

19

Notification - Audit Scope

Scope - Standards, Audit Period, Sample Size

• Standards:– Responsible for all applicable Reliability

Standards– Audit will focus on a sub set chosen by MACD– Subset reflects industry trends, impact on

reliability• Period is a rolling six years, audit team

can examine compliance during entire six years

• Data requested is usually for previous 12 months

20

Pre Audit Activities

Pre Audit Survey• A tool for the audit team to learn more

about your organization. • Generic, some questions may not apply to

your organization. • Questions about assets, organization

structure, physical location and neighbours

• Some logistical information and contact information

• Helps the audit progress more smoothly 21

Pre Audit Activities

RSAWs – Reliability Standard Audit Worksheet

• Tool to help you demonstrate your compliance

• Guides you through providing evidence to demonstrate your compliance

• Assists you in organizing and packaging the information

• Clearly identify appropriate sections in the files for evidence, reference the standard

• Submited to MACD through the Portal

22

RSAWs

23

Pre Audit Activities

• It is important to annotate documents

• If using the same document for multiple requirements, put a copy of the document in each folder!

24

Pre Audit Activities

• The IESO Portal is a secure system already in use for providing this information to the audit team

• Portal training material is available on the IESO website

• http://www.ieso.ca/imoweb/marketplaceTraining/showCourses.asp?id=15

25

Pre Audit Activities

• It is important to organize information to show where it ties to the standard (to show compliance)

• Include the RSAW for each standard in the top folder, and then evidence documentation with annotation for each requirement.

26

Pre Audit Meetings

• Open dialogue is important during the audit process. Opportunities for 3 pre audit meetings.

• Pre submission meeting – date agreed to, allows for general questions and discussion

• Pre Audit meeting, optional, done after MACD has reviewed the data submission

• Pre Audit Logistics – At least one week prior to audit to discuss the logistics of the audit week 27

On Site Logistics

Audit Team will require:• Rooms to work in• Lunch/coffee (working lunch delivered to

room)• MACD to be invoiced• Access to projector, internet, printers, if

possible via our laptops – All evidence should be electronic unless otherwise agreed by the MACD audit team

• Access to site as needed, with appropriate escorts

28

Audit Presentations

The Audit team will start the audit with a brief opening presentation.

• Introducing the team• Expectations for the process

It can be helpful if the MP makes a brief opening presentation to

• Introduce the MP audit team• Provide some background on the company and

the facility• Explain any safety/security requirements

29

Interviews

• The Audit Team may want to talk to SMEs (Subject Matter Experts) and other staff

• Your Audit Coordinator will arrange interviews

• The Audit Team may want to ask questions about procedures, their implementation, data

30

Exit Meeting

At the end of the audit period, the team will give an exit presentation. This presentation will give an overall summary:

• Recap of the audit review• The Audit Team observations, preliminary

findings and recommendations• Schedule for remainder of process,

including when the final report will be issued

31

Final Report

• The audit report would identify if the team was unable to confirm compliance with a reliability standard

• The potential non compliance is turned over to the investigation unit in MACD for follow up.

• There is no “penalty” from the audit itself

32

Potential Non Compliance Findings

• If the Audit Team identifies an immediate risk to reliability, the Team Lead will immediately communicate this to your Audit Coordinator and then to MACD for investigation

• For other potential non compliance observations that do not have an immediate impact on system reliability, these will be reported at the conclusion of the audit

33

Escalation Process

Any questions or concerns during the audit:

• Discuss with your company Audit Coordinator• The Audit Coordinator from your company

should discuss with the Audit Team Lead as soon as possible.

• If not satisfied after discussing with the Team Lead, your Audit Coordinator can escalate the issue to the MACD supervisor/director

• If still not satisfied, your company can take issues to the Dispute Resolution Panel (very formal process) – OEB process

34

Summary

• A minimum of 90 days notice• Initial Information Request - before the

audit begins, submit information to demonstrate compliance

• Subsequent Information Requests may come during the audit

• SMEs and other staff may be interviewed• The Audit Team will provide an exit

interview with preliminary results• The Audit Report will follow with details

35

Compliance Monitoring and Enforcement Audit Program -

The Audit Process

End Of Presentation

36