Post on 30-Dec-2015
description
Opinion Dynamics was contracted to conduct a primary research effort to measure Market
Participant perceptions of ERCOT’s performance with respect to meeting its responsibilities.
Results allow for comparisons between market perceptions and operational realities.
Introduction
Three Phases:
Methodology
Phase 1: 9 in-depth interviews and 2 focus groups with ERCOT staff
Phase 2: In-depth interviews with 17 Market Participants
Phase 3: Survey of ERCOT’s Market Participants
ODC developed a sample of 1,157 unique Market Participants using the following lists provided by ERCOT staff:
Appropriate points of contact at market participant firms provided by MP’s via CSR’s
ERCOT Board members from 2003 and 2004
Current Committee members from 2003-04
Attendees of the 2003 IT forum
Survey Sample
Response Rate429 completed surveys from a sample of 1,157 Market Participants (37%)
ERCOT Board Members n=16
Committee Members n=112
Market Participant Staff n=301
Classification of respondent based on self-selected descriptions – QA1
By Market Participant Firm Type
Response Rate
Consumer/Other14%
Generator/PP16%
Comp. Retailer/Rep.24%
IOU13% Co-op
11%
Power Marketer9%
Muni13%
Survey Approach: 10 point scale
Many questions based on a 10 point scale: 1-3 = negative response, 8-10 = positive response.
Mean responses will trend toward the middle of a 10 point scale – only those with passionate opinion are likely to provide a rating in top or bottom 3.
In general, mean responses of 6.6 or above are favorable ratings, 7.5 and above are extremely positive responses.
Background and ContextMarket Participant Opinions Regarding
ERCOT Staff’s Role In Developing Market Rules
Findings
ERCOT staff should participate
in market rules development
ERCOT staff should only
administer the market rules Undecided
Board Member (n=16) 38% 50% 13%
Committee Member (n=112)
67% 28% 5%
Market Participant Staff (n=301)
53% 23% 24%
Background and ContextMarket Participants’ Understanding of ERCOT’s Committee Structure
Findings
100%
0%
94%
6%
78%
22%
0%
20%
40%
60%
80%
100%
Board Member (n=16) Committee Member(n=112)
Market Participant Staff(n=301)
Group Respondent Feels is Most Responsible for Introducing Market Changes
Market Participant Committees ERCOT Staff
Background and ContextMarket Participants’ Understanding of ERCOT’s Committee Structure (cont.)
Findings
81%
19%
0%
43%
55%
2%
37%
52%
11%
0%
20%
40%
60%
80%
100%
Board Member (n=16) Committee Member(n=112)
Market Participant Staff(n=301)
Group Respondent Feels is Most Responsible for Approving Market Changes
ERCOT Board Market Participant Committees ERCOT Staff
Background and ContextMarket Participant Interactions with ERCOT Staff
Findings
CSR’s Subject Matter Expert
ExecutiveMgt.
Board Member (n=15) 27% 20% 47%
Committee Member (n=112)
38% 48% 4%
Market Participant Staff (n=299)
66% 16% 1%
Overview of Perceived Strengths
Performance of ERCOT staff, officers and directors
Personalized contact with Market Participants-- particularly CSR contact
Timeliness and accuracy of data provided
Providing effective training
Findings
ERCOT Staff Performance: Corporate Objectives(10 point scale, means shown)
Findings
Ensuring Reliability/Adequacy
of Grid
6.8
6.9
7.9
8.0
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
Nondiscriminatory access to transmission/
distribution
Accurate accounting of electric production &
delivery
Timely information about customer’s
choice of REP
Corporate objectives: Non-discriminatory access/Market Participant Registration
(10 point scale, means shown)
Findings
Level of knowledge/ expertise displayed by the client service rep
Communication and distribution of necessary information and forms
Following procedures/ protocols for market
participant registration
Board Member (n=16)
6.9 7.1 7.5
Committee Member (n=112)
7.4 7.8 8.0
Market Participant Staff (n=301)
7.9 7.9 7.9
ERCOT Staff Performance(10 point scale, means shown)
Findings
7.16.9
7.0
7.17.0
7.27.2
7.47.3
7.37.6
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
ERCOT Officers & Directors ERCOT Staff
Consistency
Attitude
Industry expertise
Responsiveness to Market Participants
Overall Performance
Management of ERCOT organization
(Officers & Directors only)
ERCOT Staff Performance: CSRs(10 point scale, means shown)
Findings
Timely response
Knowledge/ Industry Expertise
Response accuracy
Direction of inquiries
Accessibility
Attitude/Willingness to resolve problem
Overall expectations
7.4
7.6
7.6
7.7
7.8
7.8
8.2
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
ERCOT Staff Performance: Functional Areas(10 point scale, means shown)
Findings
6.3
6.6
7.2
7.4
7.5
7.7
7.9
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
Market Participant Registration
Systems Testing
Retail Transaction Processing
Scheduling
Grid Operations
Settlements and Billing
Settlements Dispute
Resolution
Communications(10 point scale, means shown)
Findings
Amount of information
Clarity of ERCOT Staff’s Messages
Written Communication
Verbal Communication
6.6
6.7
6.7
6.8
6.9
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
Timeliness
6.6
6.9
7.2
7.3
7.5
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
Communications: Functional Areas(10 point scale, means shown)
Findings
Asset Registration
Systems Changes
Progress on Market Projects
Bidding
Systems Planning
5.9
6.2
6.3
6.5
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
Communications: Room for Improvement(10 point scale, means shown)
Findings
Impacts of PRRs
Grid ops decisions
When behind schedule for
systems changes
When systems are down
Communicating…
Timeliness & Accuracy of Data(10 point scale, means shown)
Findings
6.7
6.8
7.3
7.2
7.57.3
6.97.5
6.8
7.6
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
Accuracy Timeliness
Settlements Bill
Transmission Congestion Rights
Renewable Energy Credits
Data Extracts
Metered Data
Effectiveness of Training (10 point scale, means shown)
7.2
7.3
7.3
0 1 2 3 4 5 6 7 8 9 10
SystemOperationsSeminars
Settlement andDispute Seminars
Retail TrainingSeminars
Overview of Areas for Improvement Portal reliability
Spending priorities
Systems and tools for communicating with the market
• Website navigation
• EMMS
• IT Technical Helpdesk
Functional Performance
• congestion management
• data extracts
• settlement dispute resolution Findings
6.0 5.6
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
(10 point scale, means shown)
ERCOT PortalLevel of Agreement with Statements About ERCOT IT Systems/Staff
Findings
The Portal is effective The Portal is reliable
5.7
5.8
6.2
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
ERCOT Spending Practices(10 point scale, means shown)
Findings
Spending Funds Equitably
Spending Funds on Things that are
Important to Your Company
Spending Funds Cost Effectively
4.2
5.5
5.5
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
Communications with the Market(10 point scale, means shown)
Findings
Navigation of the Web Site
Usefulness of IT Help Desk
Understanding EMMS
5.7
6.0
6.0
6.1
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
ERCOT Functional Performance(10 point scale, means shown)
Findings
Timely and Effective Implementation of Systems Changes
Timely Resolution of Settlements Disputes
Providing Data Extracts with Necessary
Content
Addressing Congestion Management issues
Market perceived areas of strength:
Grid reliability
Systems are providing timely and accurate data
ERCOT staff performance including personal interactions
Conclusions
Market perceived need for system improvements with:
Web site navigation
Data extracts (content)
Web portal
EMMS
Conclusions
Market perceived communication gaps in:
Impacts of PRRs
Systems changes
Spending priorities
Grid operations decisions
Conclusions (cont.)
Specific areas for strategic consideration by ERCOT Board:
Role of committees in setting spending priorities and introducing market changes
Defining and communicating ERCOT staff’s market function
Market is interested in more training--More research on potential types of training seminars is necessary
Conclusions (cont.)