Post on 19-Jan-2017
Utility Customer Experience Measurement – Closing the Loop April 12, 2016
Shaikat Sen, Senior Vice President, Blueocean Market Intelligence
Larry Simpson, Research Principal, Pacific Gas and Electric Company
© 2016 Blueocean Market Intelligence 1
Agenda
© 2016 Blueocean Market Intelligence 2
From Rating Scales to Rate Cases: The case for CEx Measurement in the Utility Industry
CEx 360 – A Holistic Approach
Identifying Touchpoints/Pain Points
Monitor/Track Performance
Segmentation/Modeling
Cadence
The Utility Perspective
Final Thoughts
From Rating Scales to Rate Cases: The Case for CEx Measurement in the Utility Industry
© 2016 Blueocean Market Intelligence 3
Higher Levels of Customer Satisfaction Positively Impact Credit Ratings, Operating Margins, and ROE
© 2016 Blueocean Market Intelligence 4
Higher Customer Satisfaction
In 2005, S&P clarified that their methodology for determining risk and credit ratings for utilities would include the customer satisfaction score, along with a host of other metrics
Credit Ratings
In 2011, J.D. Power found “regulated US Utilities that achieve the highest customer satisfaction in J.D. Power studies also reported the highest rates of net operating margin to FERC”
Operating Margins
J.D. Power and Associates also found that higher customer satisfaction scores were correlated with more favorable rate case decisions
Return on Equity
Expectations are Increasing, but Meeting Expectations Can Strengthen the Future
© 2016 Blueocean Market Intelligence 5
• Unprecedented impact of social media
- Customer satisfaction has risen to the top of consumer consciousness
• Public exposure to benchmarking studies (J.D. Power)
- Customers are seeing how other utilities are being perceived by their customers
• Customer satisfaction as a competitive differentiator
− Critical when seeking growth through the development of new unregulated products and services, or protecting the core business from disruptive entrants
• Better customer experiences
− Cornerstone for positive outcomes around co-created energy efficiency solutions
Utilities Ranked 9 Out of Top 14 Industries on Customer Satisfaction, Trust in Utilities is on the rise.
In a worldwide study conducted in 2014, Accenture found that trust in utilities had increased by 13% to 37% in 2014…up from 24% in 2013.
© 2016 Blueocean Market Intelligence 6
Expectations from Utility Companies (2014)
8.59
8.52
8.41
8.35
8.28
8.20
7.89
7.88
7.82
7.73
7.71
7.71
7.66
7.55
Customer Engagement | McKinsey on Marketing & Sales
1 Brokerage
2 Auto Ins
3 Retail
4 Bank
5 Hotel
6 Health Providers
7 Mobile Phone
8 Airlines
9 Utilities
10 Phone
11 Postal
12 Health Inc
13 Internet
14 Pay TV
Rank 2012
Industry CSAT
average score1
2011-2013 trend
1 Customer satisfaction was measured on a scale of 1-10; includes top 3 companies per industry per respondent. Consistency of experience operationally defined as standard deviation, that is, consistency of perception across surveyed demographic
Simple home improvement products and materials to
save energy
66%
Home energy audits/consultations to help save
energy
62%
Home energy generation products 58%
Devices or services to automate home energy
management
56%
Installation and/or maintenance services for home
energy devices
56%
Backup energy storage or power generator in case
energy goes out
53%
Warranty and/or financing plans for home energy
devices
52%
Source: The New Energy Consumer: Architecting for the Future, Accenture, 2014.
Components of an Effective 360 CEx Measurement Program
© 2016 Blueocean Market Intelligence 8
Insights
Information Sources to Tap Into
© 2016 Blueocean Market Intelligence 9
• Qualitative • Explore customer journeys • Identify touchpoints • Assess pain points
• Quantitative • Relationship surveys • Transactional surveys
• Blogs/forums/discussions
• Complaint/review web sites
• Facebook/Twitter/other sites
• Segmentation data
• J.D. Power results
• Other benchmarking studies
• Rate plan participation
• Bill payment history
• Energy usage
• Demographics
• Call center transcripts
• Customer emails
• Customer/field service agent updates
Mapping Information Sources to the 360 CEx Cycle
© 2016 Blueocean Market Intelligence 10
Implement Changes
Communicate with Customers
Segmentation/ Modeling
Transactional Data/ 3rd Party Data
Identify Touch/ Pain Points
Qualitative Research/ Unstructured
Feedback/Social Media
Monitor/Track Performance
Quantitative Surveys/ Benchmarking Surveys
Harvesting Information…
© 2016 Blueocean Market Intelligence 12
Enumerate, identify and map touchpoints through customer focus groups and journey mapping sessions with internal stakeholders
Monitor relevant industry news through clipping services, RSS feeds, etc.
Systematically listen to and harvest information from as many customer information sources as possible
Recordings Transcripts Emails Field Notes
Adopt one or more social media monitoring tools to “scrape” data from a multitude of social media websites
Invest in text analytics solutions to analyze unstructured data streams
…And Harnessing It
© 2016 Blueocean Market Intelligence 13
Siphon all feedback to a dedicated “Listening Intelligence Team”
Code and monitor incoming information on an ongoing basis
Develop a standardized code frame to organize unstructured information across all data sources
Generate ongoing weekly/monthly reports highlighting top customer issues
Visualization Examples of Unstructured Data
© 2016 Blueocean Market Intelligence 14
Theme Comment
Overall Service/ Feedback (+)
• No problems/very few power outages/service is good/happy with service
• Good company/honest/give back to community
Overall Service/ Feedback (-)
• Frequent outages/Service is not reliable
• Service is average
• Not aware of any other services that are provided
• There is always room for improvement/no one is perfect
• Do not like company
• Heard negative things about company
• Not happy or dissatisfied/neutral
Customer Service/ Field Service(+)
• Customer service (friendly, helpful, knowledgeable, etc.)
• Respond quickly to service issues
Customer Service/ Field Service (-)
• Customer service (hard to reach reps, not helpful, etc.)
• Do not respond quickly to service issues
Billing/ Website mention (+)
• Accurate billing/timely
• Online billing available
• Easy to pay bills/easy to read bills
• Can see/compare energy usage
• Website mentions/easy to use
Billing/ Website mention (-)
• Billing issues
• Website issues (confusing, no online billing)
Rates/Discounts (+) • Lower rates/discounts
Value of Quantitative Measurement
© 2016 Blueocean Market Intelligence 16
Quantitative Assessment is the Backbone of a Strong Measurement Program
• Results are representative of your overall customer base
• Identifies specific areas where you are doing well and those needing attention
• Helps to prioritize your organizational focus to get the best ROI
• Perception is reality – you need to keep your finger on the pulse of your customers’ perceptions
Components of a 360 Program
• Transactional surveys
• Relationship surveys
• Competitive benchmarking (optional)
Part 2: Develop transactional surveys associated with each customer journey
Part 1: Understand the customer journey associated with each transactional touchpoint
Transactional Surveys – Best Practices
• Conduct customer journey focus groups
− Internal multi-functional stakeholder brainstorming sessions
− Customer focus groups
• Enumerate as many customer journeys that reflect how your customers interact with your organization
− New customer sign-up
− Tree trimming
− Emergency service response
− Other
• Prioritize customer journeys and develop a strategy and timeline to address each journey
• Keep the survey short
• Focused on the specific customer journey
• Include questions that tie back to high level customer satisfaction metrics
• Deploy after each transaction is complete
© 2016 Blueocean Market Intelligence 17
Relationship Survey Design – Best Practices
© 2016 Blueocean Market Intelligence 18
Characteristics of High Performing Utilities
Positive customer experience
High levels of customer engagement in products and services offered
Strong communication with customers
High levels of customer trust in the brand
Overall Experience
Overall Trust
Overall Communication
Engagement Levels
Ease of Doing Business
Overall Satisfaction
Likelihood to Recommend
Likelihood to Switch
Relevant Key Metrics to Consider
Experience, Trust, and Communications – Example Metrics
© 2016 Blueocean Market Intelligence 19
Billing
Payment options
Billing statement clarity
Billing statement accuracy
Meter accuracy
Pricing
Fair price
Education - rate plans
Timely change notifications
Custom rate plans
E-Efficiency
Education - saving energy
Energy efficiency incentives
Competence
Process excellence
Well-managed
Leader
Corporate Values
Keeps promises
Take responsibility
Open communications
Cares about employees
Customer First
Values customers
Extra mile
Ease of doing business
Understands needs
Customer Experience Customer Trust
Communications
Other Communications
Web site assessment
Specific marketing collaterals
Advertising recall
External News
Exposure to news
Favorability/un-favorability of news
Word of Mouth
Discussions with others
Positive/Negative opinions shared
Prioritizing Actions – Best Practices
• Carefully plan the structure and sequence of the questionnaire
• Determine where you want the key metrics to be:
© 2016 Blueocean Market Intelligence 20
• Keep attributes and related dimensions together, but randomize and rotate dimensions and attributes to prevent order bias
• Decide on an appropriate rating scale (5-pt., 7-pt., 10-pt.) and stay consistent
• Establish a theoretical model of how attributes might relate to each dimension, and how each dimension relates to the key outcome measure
• Responses sensitized by preceding assessments
• Statistical models tend to be stronger
• But, results may not reflect reality
…or at the end
At the beginning…
• Responses tend to reflect top-of-mind perceptions
• Results may be more closely aligned with real life
• However, statistical models tend to be weaker
Values customers
Hypothesizing Expected Relationships
© 2016 Blueocean Market Intelligence 21
Overall Satisfaction
Trust
Experience
Customer First
Environment
Competence
Billing
E-Efficiency
Reliability
Management
Leadership
Payment options
Statement clarity
Statement accuracy
Extra mile
Ease of doing business
Education on saving energy
Energy efficiency initiatives
Process
Driver Modeling – Best Practices
© 2016 Blueocean Market Intelligence 22
Standard regression doesn’t work well with attitude and perception data
New techniques are available today to get around these challenges
• Shapley Value Analysis or Dominance Analysis or Relative Weight Analysis
• Impractical to use before because of the enormous amount of computing power required
• Today’s computers are able to run the required hundreds of thousands of regressions in a matter of hours
• Traditional techniques can fail when predictor variables are correlated with one another or have skewed distributions
• Attitude and perception data are almost always correlated and skewed
Relative Importance of Drivers
© 2016 Blueocean Market Intelligence 23
Standard Regression Model Shapley/Dominance Analysis
R2= 60%
Var. Important %
Predictor 1 = 15%
Predictor 2 = 20%
Outcome - dependent var.
Predictor 1 , Predictor 2 - Independent var.
Predictor 1
Predictor 2
Outcome
40%
15% 25%
20%
Multicollinearity
Outcome - dependent var.
Predictor 1 , Predictor 2 - Independent var.
Predictor 1
Predictor 2
Outcome
40%
15%
20%
Multicollinearity
25%
21% 4%
R2 = 60%
Var. Important %
Predictor 1 = 36%
Predictor 2 = 24%
2%
7% 11%
21%
25%
34%
Field ServiceCustomerService
BillingE-EfficiencyReliabilityPricing
Relative Importance
10%
16% 16% 18%
23%
26%
CorporateValues
EnvironmentCompetenceCommunityCustomerFirst
Safety
Relative Importance
Overall Satisfaction
Customer Trust 40%
Customer Experience 60%
R2=80% R2=70%
© 2016 Blueocean Market Intelligence
Relative Importance of Dimensions
24
Customer Experience 60%
Pricing E-Efficiency Reliability Billing Customer Service
7% Field Service
11%
14%
16%
17%
19%
25%
Customer Service Representative
Problem resolution
Overall time
CSR Competence
CSR Communication
CSR Courteousness
Phone system navigation
Phone system navigation
Overall time
Problem resolution
R2 = 78% R2 = 80%
Live (70%) Automated (30%)
31%
30%
39%
Automated CustomerService
© 2016 Blueocean Market Intelligence
Relative Importance of Attributes
25
Sampling – Best Practices
© 2016 Blueocean Market Intelligence 26
Target Audience
• Residential surveys – head of household/spouse/partner, etc.
• SMB surveys -
One account can have multiple sites; same company can have multiple accounts
Ideally, your population should be defined as an account and site combination that has a unique account contact
• Minimum recommended sample size for a segment is about 400
• However, you have to consider all relevant segments
• Recommended minimum sample size for each sub-segment is 100 – or more – if budget allows
Sample Size Sample Sizes and Error Margins
Margin of error (Precision)
Confidence Level
90% 95% 99%
3.0% 747 1067 1849
5.0% 269 384 666
10.0% 67 96 166
Data Collection Options
© 2016 Blueocean Market Intelligence 27
Advantages Disadvantages
Channels
Outbound Phone
Inbound Phone/ SMS Surveys
Regular Mail
Web-based
App-based
• Expensive • Cellphone prevalence making it more and
more difficult and expensive • Refusal rates are increasing • Tends to skew towards older, women
• IVR programming required (inbound phone) • Need to automate survey offering to
prevent bias (inbound phone) • Surveys need to be very short
• Response rates are very low • Need to have email addresses
• More expensive than email • Very low response rates • Usually takes longer in field
• Requires internet awareness and may skew sample to a younger demographic
• Customer has to come to the web site to get an opportunity to participate
• Still in infancy, but gaining popularity • Low uptake for older generation
• Easy to complete • Easy to control • Relatively high response rate
• Immediate response – feedback is more accurate
• Generally good response rates • Inexpensive after initial set-up
• Relatively cheap and easy to set up • Cost effective to run continuously • 80-90% of responses are in within 72 hours
• Generally cheaper than telephone • Can capture good, detailed feedback
• Often used to ‘rate’ content • Useful for web site assessments • Typically used in conjunction with email
surveys
• Emerging approach • Cost effective to run continuously • “In”
Data Collection – Best Practices
© 2016 Blueocean Market Intelligence 28
Mixed/hybrid surveys (landline, cellphone, and email) may be optimal
– Acceptable for ongoing tracking programs
– Allows you to capture a more representative sample of your customers
– Happy medium in cost and ability to leverage existing customer information
Customer Experience
60
65
70
75
80
Q2 '14 Q3 '14 Q4 '14 Q1 '15 Q2 '15 Q3 '15 Q4 '15
Phone CONTROL Phone TEST Online TEST Combined TEST
• Ensure sample composition matches distribution of landline, cell phone, and email addresses available in the customer database.
Leveraging Transactional/Syndicated Data to Profile, Understand and Predict Customer Behaviors
© 2016 Blueocean Market Intelligence 30
Utilities have an enormous amount of data on their customers
Allows you to associate satisfaction with factors like energy use, monthly bills, household composition, and so on
Matching back quantitative survey data to customer base can yield deeper insights about attitudes and behaviors
Attitude and perception data allows modeling of behaviors and segmentation to maximize produce development and communication opportunities
Developing a Customer Engagement Metric
© 2016 Blueocean Market Intelligence 31
(selects EE programs & renewable programs)
Management of energy usage
Adoption of new technology
Takes advantage of offers
Interactions with Utility
(signs up for MyAcct, eNotifications, surveys)
(selects TOD programs, specific eNotifications)
(campaign respondents, enrolls in BPCC programs)
“Dimension-level”
engagement score
Overall “Engagement
Score” - weighted average
of dimensional engagement
scores
50.30%
12.90% 17.60% 19.20%
Adopt NewTechnologies
Interaction withUtility
Manages energyusage
Takes Advantagesof Offer
Composite dimension weights
The Customer Engagement Continuum
© 2016 Blueocean Market Intelligence 32
Focus Areas
.00 13.90 41.70 69.51 97.31 125.11 152.91 180.70 208.52 236.32 264.12 291.93 319.73 347.53 375.33 403.14 27.80 55.60 83.41 111.21 139.01 166.81 194.62 222.42 250.22 278.02 305.83 333.63 361.43 389.23 417
Poorly engaged Moderately engaged Highly engaged Very Highly engaged
Poorly engaged
Moderately engaged
Highly engaged
Very highly engaged
Interaction with Utility
Managing Energy
Takes Advantages of offers
Adopt new technology
Low Medium High
2% 13%
26%
54%
5%
Very highlyengaged
customers
Highly Engagedcustomers
Moderatelyengaged
customers
Poorlyengaged
customers
Non-engagedcustomers
Segment Size
Type of Dwelling (Single Family)
Definite Home Owner
Household Age (>45 yrs.)
Months at Residence (>82 mths.)
House Square Footage (>1920 sqft)
Pool at home
Pool Area (>100 sqft)
Home Construction quality (good)
Customer tenure (10+ yrs.)
HH Income Range ($75K+)
Married (Husband & wife) with/without child
Number of People in HH (4+)
Demographics, Attitudes, Energy Use
© 2016 Blueocean Market Intelligence 33
Ho
me
92%
84%
71%
63%
49%
32%
32%
31%
69%
49%
48%
24%
Total
77%
64%
61%
41%
36%
18%
18%
18%
52%
32%
33%
17%
Ho
use
ho
ld
1%
1%
2%
3%
4%
4%
6%
7%
9%
10%
12%
19%
22% Green Investors
Payback Investors
Green Activists
Show-me Participants
First Costers
Hands-On Believers
Pragmatists
Unplugged
Creatures of Comfort
Budget Watchers
Living in the Now
Tech Frontiersmen
Tech to Live
22%
26%
34%
40%
62%
65% Affordability level
Green affinity
Info action orientation
Investment capacity
Technology propensity
Comfort consumption
13%
12%
9%
11%
17%
5%
4%
10%
4%
5%
6%
2%
5%
40%
52%
26%
23%
26%
15%
Ener
gy C
on
sum
er D
ynam
ics
Axc
iom
Var
iab
les
Total
1440.3 1159.5 274.1 $161.1
Total (avg.) OffPeak (avg.) OnPeak (avg.) Enrgy Bill (avg.)
kWh
kWh
1231 kWh 902 kWh 170 kWh $142 Total
Last 12 Month Energy Use and Bill
Total (avg) Off Peak (avg) On Peak (avg) Energy Bill (avg)
Engagement Segment Profiling
© 2016 Blueocean Market Intelligence 34
Very Highly Engaged Average Engagement Score: 318/1,000
Higher income, likely to spend more and invest more, stay in larger houses with pools. Innovators by nature and pro-active. Display high affinity for comfort consumption and environmental friendliness.
Size: 1.8%
Highly Engaged Average Engagement Score: 191/1,000
Likely to be home-owners who are financially stable and have high disposable incomes. Tend to be older customers. In general they also have an affinity for comfort consumption and environmental friendliness.
Size: 13%
Moderately Engaged Average Engagement Score: 110/1,000
Average income, middle aged families with children staying with more no. of people in HH. Have a pragmatic approach and high info/action orientation. Tend to be innovative, green friendly, and prefer comfort consumption.
Size: 26%
Poorly Engaged Average Engagement Score: 43/1,000
Tend to be newer customers with less energy consumption having poor credit histories, staying in smaller apartments. Mainly young single individuals, tech savvy and mobile, yet to be anchored in long term housing . Generally have lower household incomes.
Size: 54%
Not Engaged Average Engagement Score: 0/1,000
Single wanderers – prefer living in apartments/town houses and usually not for more than 2 years. Financially cautious with a lower propensity for technology and green friendly initiatives. Very low on comfort consumption.
Size: 5%
Customer Experience Measurement - Cadence
© 2016 Blueocean Market Intelligence 37
Month
1 2 3 4 5 6 7 8 9 10 11 12
Ongoing
Unstructured (text) analysis
Transactional Surveys
Relationship Surveys
Ad Hoc Deep Dives/Drill Downs
Behavioral Connections/Modeling
Dip-stick Quantitative Surveys
Qualitative Deep Dives/Ethnographics
Competitive Benchmarking Survey
Utility Perspective
© 2016 Blueocean Market Intelligence 39
Evolve research to changing environment while maintaining essential trending
Adapt to measure emerging customer needs and obtain feedback about major events
Complement syndicated research with proprietary surveys
Own survey content, manage changes to methodology, identify respondents for data appends and segmentation, and holistically measure gas and electric satisfaction
Partner with data science to set key goals and forecast performance
Process has grown from analyzing trend data and using management judgement to employing advanced statistical modeling and Monte Carlo simulation
Supplement perception research to add clarity
Deep dive into customer touch points using transactional surveys, qualitative research, and operational data
Utilize key research to inform the Customer Experience (CX) team
Collaborate to measure and analyze data, identify and prioritize friction points, work cross-functionally, and inform strategic decision making
Final Thoughts
© 2016 Blueocean Market Intelligence 40
Leverage existing assets (harness all available unstructured/internal data)
Identify a relevant and comprehensive set of attributes to measure
Ensure that the instrument is designed to allow for statistical analysis of key drivers of overall satisfaction
Create enthusiasm and accountability within your organization
Deploy transactional surveys to monitor customer journeys
Develop a tracking survey instrument that can remain stable over a period of one to three years
United States | United Kingdom | India | United Arab Emirates www.blueoceanmi.com
Thank You
Shaikat Sen
Senior Vice President
ssen@blueoceanmi.com
Larry Simpson
Research Principal
Pacific Gas and Electric Company
© 2016 Blueocean Market Intelligence 41