“A new customer experience measurement model – a meta analytical review of findings over the...

In partnership with:

A new Customer Experience Measurement

Model – A Meta Analytical Review of Findings

over the period 2002 to 2009

Presented by Prof Adré Schreuder

MD of Consulta Research & Extra-ordinary Professor

of Marketing Research – University of Pretoria,

South Africa

19th Annual Frontiers in Service Conference 2010

10-13 June 2010 - Karlstad, Sweden

Index

• Background & Rationale for Research

• Previous Research and Literature Review

• Research Question & Objectives

• Research Methodology & Data Analysis

• Research Results & Discussion – Dangers of Reporting Net Measures in Isolation

– Satisfaction Measures as Predictors of NPS

– Normality of Customer Experience Modelled Score

• Conclusion

Slide 2

http://www.consulta.co.za/

Background & Rationale for Research

Slide 3


Terminology Confusion

Slide 4

Source: Created by Adré Schreuder – reference:

< http://www.wordle.net/show/wrdl/1954142/Customer_experience >


http://www.wordle.net/show/wrdl/1954142/Customer_experience

CUSTOMER SATISFACTION

A Historic Overview

Slide 5

• TQM of Edwards Deming - Zero Defect, Six Sigma

Relationship

Quality Era

(1995)

CRM

Customer

Experience Era

(2003)

CEM

Service Quality

Era (1984)

SERVQUAL

Product Quality

Era (1950’s)

TQM

• The Nordic approach (Grönroos 1984: Technical/Functional Model, Lethinen & Lethinen 1988 : Technical, Corporate, Interactive)

• The North American Debate (PZB 1985: SERVQUAL (Gap-based measure, Familiar five quality dimensions, Cronin & Taylor 1992: SERVPERF - Performance only measure, Brown Churchill & Peter 1993: Better/worse than expected scale, Teas 1993: Evaluated Performance Model = gap between perceived performance & ideal amount of feature)

• Jagdish Sheth introduced Relationship Management in mid 90’s

• Growth of CRM-systems and popularity

• NPS introduced by Reichheld in 2003 – CEM era is born


Customer satisfaction:

Contrasting academic and consumers’ interpretations

Satisfaction defined

– Derived from Latin “satis” = enough & “facere” (faction) = to do/to

make

– Early interpretation and use of the word mostly focused on “some sort

of release from wrong doing” - later “release from uncertainty”

• At least two basic approached in defining the concept:

– CS viewed as an outcome of a consumption activity

– CS viewed as a process

• Most widely adopted description = evaluation between what was

received and what was expected

Slide 6

Source: Parker, C & Mathews, B.P. Marketing Intelligence & Planning, 19/1 2001 (pp 38-44)



CS viewed as an outcome - Focus on the nature (not cause) of

satisfaction:

• Emotion - satisfaction is the surprise element of product

acquisition and/or consumption experiences, or an affective

response to a specific consumption experience

• Fulfilment - motivation theories state that either people are

driven by the desire to satisfy their needs or achieving specific

goals.

• State - Oliver’s (1989) framework of four satisfaction states,

where satisfaction is related to reinforcement and arousal.

– Low arousal = “satisfaction-as-contentment”

– High arousal = “satisfaction as surprise” (positive / delight or

negative / shock)

– Positive reinforcement = “satisfaction-as-pleasure”

– Negative reinforcement = “satisfaction-as-relief”

Slide 7




CS viewed as a process

• Concentrate on the antecedents to satisfaction rather than

satisfaction itself. (Origins in discrepancy theory - (Porter, 1961)

and Contrast Theory (Cardozo, 1965);

• Most common interpretation = a feeling which results from a

process of evaluating what was received against that expected, the

purchase decision itself and/or the fulfillment of needs/wants.

• Most “well-known’’ descendent of the discrepancy theories is the

expectation disconfirmation paradigm (Oliver, 1977, 1981).

Slide 8



Customer Experience – the new “Customer

Satisfaction”?

• “Yet despite the recognition of the importance of customer

experience by practitioners, the academic marketing literature

investigating this topic has been limited.

• Publications on customer experience are mainly found in

practitioner-oriented journals or management books … tend to

focus more on managerial actions and outcomes…

• The literature in marketing, retailing and service management

historically has NOT considered customer experience as a

separate construct. Instead researchers have focused on

measuring customer satisfaction and service quality.”

Source: Verhoef, Peter C., Katherine N. Lemon, A. Parasuraman, Anne Roggeveen, Michael Tsiros and Leonard A.

Schlesinger (2009), “Customer Experience Creation: Determinants, Dynamics and Management Strategies,” Journal of

Retailing, 85 (1), 31–41.

Slide 9


Customer Experience – the new “Customer

Satisfaction”?

• One reason for the apparently weak observed link between

satisfaction and future behaviour may lie in the role of emotions

• Previously studies emphasised cognitive aspects of satisfaction –

growing body of evidence that affective measures of satisfaction

(which incorporate emotions) may be a better predictor of

behaviour

• As a cognitive measure, satisfaction is more likely to be distorted

over time than a measure that incorporates an affective

component (emotions are more deep-seated & more stable over

time)

• Satisfaction should thus include a combination of an evaluative

(cognitive) and emotion-based (affective) response to a service

encounter

Source: Koenig-Lewis, N. and Palmer, A. "Experiential values over time – a comparison of measures of satisfaction

and emotion," Journal of Marketing Management (24:1-2), 2008, pp. 69-85.

Slide 10


Construct definition of “Customer Experience”

• The customer experience construct is holistic in nature and

involves the customer’s cognitive, affective, emotional, social and

physical responses to the retailer.

• This experience is created by:

– controllable elements - service interface, retail atmosphere,

assortment, price,

– uncontrollable elements - influence of others, purpose of shopping

• Customer experience encompasses the total experience, including

the search, purchase, consumption, and after-sale phases of the

experience, and may involve multiple retail channels.

• Three major focus areas:

– cognitive evaluations (i.e., functional values)

– affective (emotional) responses

– social and physical components

Slide 11

Source: Verhoef, Peter C., Katherine N. Lemon, A. Parasuraman, Anne Roggeveen, Michael Tsiros and Leonard A.

Schlesinger (2009), “Customer Experience Creation: Determinants, Dynamics and Management Strategies,” Journal of

Retailing, 85 (1), 31–41.


Putting Customer Experience into Perspective

• The term Customer Experience Management

is used within the broader context of

Customer Relationship Management (CRM) –

clearly seen in the view of Kirkby, Wecksell

& Janowski (2003) when they say: “CEM is

part of customer relationship management

(CRM) and the natural extension of building

brand awareness.

• Where brand gives the promise, CEM is the

physical delivery of that promise and is vital

in an economy where a brand is increasingly

built on value delivered rather than product

features”.

Slide 12

Illustration Copyright – Consulta 2010


Putting Customer Experience in Perspective

Slide 13


Index








• Conclusion

Slide 14


Previous Research & Literature Review

• Collection of previous research and literature regarding

Customer Experience measurement are presented and

discussed under the following topics: – Multi-attribute measures such as:

• SERVQUAL,

• ASCI &

• Others – Effort Score & ERIC

– Net Measures such as: • The Net Promoter Score from Fred Reichheld & Bain Company

• Secure Customer Index from Burke

Slide 15


Customer satisfaction and company

profitability: The Service-Profit Chain

Slide 16

External

Service

Value

Profitability

Internal

Service

Quality

Employee

Satisfaction

Employee

Retention

Employee

Productivity

Customer

Satisfaction

Revenue

Growth

Customer

Loyalty

3Rs (>Market Share) • Retention, • Repeat Business • Referrals

Service designed & delivered to meet targeted customer’s needs

Service Concept: Results for Customer

• Workplace Design • Job Design • Employee Selection & Development (skills

& empowerment drives good feelings towards the firm)

• Employee Rewards & Recognition • Tools for Serving Customers

Operating Strategy & Service Delivery System

Adapted from: Heskett, Jones, Loveman, Sasser & Schlesinger (HBR – 1994, HBR July/Aug 2008, p.120)


The GAP never mentioned …

Slide 17

CEM

– T

he “

Mis

sing G

ap”

Expectations

Perceptions

Delivery

Interface

Management

understanding of

expectations

Marketing &

Communication

Experience

Standards

Gap 1

Gap 2

Gap 3

Gap 4

Gap 5

CEM = delivering what our

customers expect us to – and

a little bit more ‐,

making them feel great at

every “moment of truth”,

Adapted from original Gaps-

Model of Parasuraman,

Zeithaml & Berry Illustration Copyright – Consulta 2010


CEM

Str

ate

gy

Conceptual Model of Customer

Experience Creation

Slide 18

Source: Verhoef, Peter C., Katherine N. Lemon, A. Parasuraman, Anne Roggeveen, Michael Tsiros and Leonard A. Schlesinger (2009),

“Customer Experience Creation: Determinants, Dynamics and Management Strategies,” Journal of Retailing, 85 (1), 31–41.

Social Environment: Reference group, tribes, co-destruction, service staff

Service Interface: Service person, technology, co-creation/customisation

Retail Atmosphere: Design, scents, temperature, ambient noise, music

Assortment: Variety, uniqueness, quality

Price: Loyalty programs, promotions, rewards

Customer experiences in alternative

channels

Retail Brand

CUSTOMER EXPERIENCE (t – 1)

Situational

Moderators: Type of store, location,

culture, economic climate,

season, competition

Consumer

Moderators: Goals: experiential

Task orientation, socio-

demographics, consumer

attitudes (price sensitivity,

involvement)

Customer Experience

(t): Cognitive, affective, social,

physical


Effort Score – worth the effort?

Slide 19

Pre

dic

tive

Po

we

r* o

f

Rep

urc

has

e

High

Low

Low High Predictive Power* for Increased

Spend

Power* - Linear regression coefficients regressed against Likelihood

to Repurchase & Increase Spend

Research conducted by Customer Contact Council of the

Corporate Executive Board

NPS® Council Conclusion “Inadequate measure” in the service channel: • Question inherently positive

(only likelihood to recommend – not criticize)

• Captures company-level sentiment (incl brand, product, pricing)

Effort

Council Conclusion Better suited for service channel. Better financial predictor & best indicator of loyalty

CSAT Council Conclusion Popular, widely used BUT “not sufficient in predicting financial outcomes … de-emphasize its use in strategic decisions”

Comments:

• Directly contrasting scientific

proof of ACSI (American), SCSI

(Sweden)

• No scientific foundation

• Irresponsible to “recommend”

members against

• Effort-score purely developed in

Contact centre environment

• No published proof of scientific

reliability & validity

• Scale is reverse scored – South

African research shows low

reliability & poor predictive

properties to the contrary


ERIC™ – Empathy Rating Index

Slide 20

Source: Lywood, J., Stone, M. and Ekinci, Y. "Customer experience and profitability: An application of the

empathy rating index (ERIC) in UK call centres," Journal of Database Marketing & Customer Strategy

Management (16), 2009, pp. 207-214. & Lywood, J., Stone, M. and Hackett, D. Eric Methodology Whitepaper

2005 < http://www.empathy.co.uk/ >

• The ERIC instrument consists of 29 empathy questions measured

on a 10-point rating scale and 11 call process questions that are

related to how the calls are processed

• The trained researchers (mystery callers) then make 40

unscripted(?) calls over three weeks to each company and

complete an online questionnaire

• The study sample was limited to 28 companies in which ROCE and

ERIC ratings were both available.


http://www.empathy.co.uk/

ERIC – Testing the claims

Slide 21

Comments:

• No proven scientific grounding

• Non rated Journal, 6 rated references

used

• Questionable statistics & sample

• No longitudinal data or reference to

time

• Methodology basically mystery caller

• Psychometric properties of scale – no

scientific grounding

• Mixed construct in scale (15 constructs

across 33 statements

• Of 5 attributes only one (Empathy) is

an interval scale, all other “Yes/no” or

numerical (number of calls)

• Claimed at 2008 CS Conference = False

claim

Source: Lywood, J., Stone, M. and Ekinci, Y. "Customer experience and profitability: An application of the

empathy rating index (ERIC) in UK call centres," Journal of Database Marketing & Customer Strategy

Management (16), 2009, pp. 207-214. & Lywood, J., Stone, M. and Hackett, D. Eric Methodology Whitepaper

2005 < http://www.empathy.co.uk/ >

Claimed at 2008 CS Conference:

“At Last –a proven link between a service

related measure and profitability”


http://www.empathy.co.uk/

Net Promoter Score – single net measure

• A simple recommend question measured on 0 to 10 scale of

likelihood to recommend

“How likely is it that you would recommend (brand or company X)

to a friend or colleague?”

• Net Promoter score is calculated by taking the percentage of

“promoters” (9-10 rating; extremely likely) and the percentage of

“detractors” (0-6 rating; extremely unlikely)

NPS = % of Promoters minus % of Detractors

• Companies with scores above 75% have world-class loyalty and

word-of-mouth, which will correlate with a firm’s growth1

Slide 22

1Reichheld, F. (2003). The One Number You Need to Grow. Harvard Business Review, Dec 2003


Net Promoter Score – single net measure

Slide 23

• NPS adopted by executives: • Swift to survey

• Simple to understand and

communicate

• Top-of-house dashboard metric

• Reichheld (2003): NPS is a more

accurate predictor of sales growth

than the elaborate American

Consumer Satisfaction Index1

• General Electrics CEO: “This is the

best customer satisfaction metric

I‟ve seen”

Positive Negative

1Reichheld, F. (2003). The One Number You Need to Grow. Harvard Business Review, Dec 2003 2Keiningham, T. et al. (2007). The value of different customer satisfaction and loyalty metrics in predicting customer retention, recommendation, and share-of-wallet, Managing Service Quality 17(4), 361-384. 3Morgan, N. & Rego, L. (2006). The Value of Different Customer Satisfaction and Loyalty Metrics in Predicting Business Performance. Marketing Science 25(5), Sep – Oct.

• Little scientific research linking

recommend intentions to actual

intentions2

• Morgan and Rego (2006) assessed

six different metrics over a seven

year period and found: “…recent

prescriptions to focus customer

feedback systems & metrics solely

on customers‟ recommendation

intentions and behaviours are

misguided”3


Testing the Net Promoter Score® claims

• Contrary to Reichheld’s assertions, the results indicate

that recommend intention alone will not suffice as a

single predictor of customers’ future loyalty

behaviour.

• Use of a multiple indicator instead of a single

predictor model performs better in predicting

customer recommendations and retention.

• Thus far, however, there have been no peer-reviewed,

scientific investigations examining the relationship

between recommend intention and customer

behaviours (outside of customer referral/complaining

behavior).

Slide 24

Source: Keiningham, T., Cooil, B., Aksoy, L., Andreassen, T. and Weiner, J. "The value of different customer satisfaction and

loyalty metrics in predicting customer retention, recommendation, and share-of-wallet," Managing Service Quality (17:4),

2007, pp. 361-384


Testing the Net Promoter Score® claims

• FINDING: “The assertion that recommend intention alone will

suffice as a predictor of customers‟ future loyalty behavior

(Reichheld NPS), however, is not supported. We reach this

conclusion based upon three primary findings.

– First, bivariate correlations of all the attitudinal variables

and customer behaviours investigated tended to be modest.

– Second, when examining the three primary behaviours

associated with customer loyalty (retention, share of

wallet, and recommendations) recommend intention was

generally not the best predictor for each of these variables.

– Third, multivariate models universally outperformed models

that use only recommend intention

Slide 25

Source: Keiningham, T., Cooil, B., Aksoy, L., Andreassen, T. and Weiner, J. "The value of different customer

satisfaction and loyalty metrics in predicting customer retention, recommendation, and share-of-wallet," Managing

Service Quality (17:4), 2007, pp. 361-384


Secure Customer Index as Net measure

• The Secure Customer Index® probes three attributes1: – the “secure” customers were very satisfied,

– had a likelihood to definitely continue using the service,

– and had a likelihood of definitely recommending the service to

others

• Customers grouped into subgroups or loyalty segments

• Direct linkage to financial & market performance was

calculated

Slide 26

1Brandt, D. (1996). Customer Satisfaction Indexing, Conference Paper presented at American Marketing Association, USA

Secure Favourable Vulnerable At Risk


Secure Customer Index (SCI) as net measure

• Today the new improved SCI® is Burke Incorporated’s proprietary

modelling approach

• Five dimensions to assist validity and predictions of future share of

wallet:

• Burke has studied data which directly links and also projects a

correlation between customer satisfaction, loyalty, and value to

financial performance

• Through projection and direct linkage, they can calculate which

part of the marketing mix will bring the largest ROI

Slide 27

Earned Loyalty

Likelihood to

Recommend

Likelihood to

Repurchase

Overall Satisfaction

Preferred Company


Customer Experience – A “deep ecological paradigm” shift (Fritjof Capra – The Web of Life, 1996)


Key Drivers of Loyalty


Outcomes of Improved Customer Experience

Outcomes of Customer

Experience

Customer-Related

Outcomes

Efficiency-Related

Outcomes

Employee-Related

Outcomes

Overall Performance-

Related Outcomes

Behavioral

Intentions

Customer

Behaviours

Customer

Commitment

Repurchase

Intentions

Price Perceptions &

Willingness to pay

Customer Loyalty &

Repurchase Behaviour Word-of-Mouth &

Complaining Behaviour

Financial

Performance

Nonfinancial

Performance

Source: Luo, X & Homburg, C. April 2007 Neglected Outcomes of

Customer Satisfaction. Journal of Marketing, Vol 71, Apr 2007 (0 133-

149)

Behavioral Intentions are determined by

how the drivers of Customer Satisfaction

are managed <by implication measured>

– this is the essence of Customer

Experience Management

Customer

Defection


Index








• Conclusion

Slide 31


Research Question

• The popularity of the Net Promoter Score has

highlighted the use of net measures in customer

experience measurement

• Considering the preceding literature review and

discussion regarding different net measures, it is

obvious that no single measure can be used

successfully in measuring the complex constructs

of customer experience, customer satisfaction

and customer loyalty

• This presentation will explore a quantitative

model that integrates the “best-of-both-worlds”

through a combined metrics of net measures and

a multi-attribute measure of customer

experience

Slide 32


Research Objectives

Slide 33

• Explore the use and application of Net Measures in the measurement of Customer Experience

• Compare Net measures in terms of reliability, validity, predictive ability and practical application

• Position Net Measures within the body of knowledge of multi-attribute Customer Experience Measurement theory and practise

The purpose of this study is to investigate the following three objectives:


Research Design & Data Collection

• Meta-analysis on data collected over a time frame of more than 5

years, covering more than 1.5 million customer interviews across

South Africa

• Survey results have been consolidated from enterprise wide

proprietary customer satisfaction surveys across a range of clients

• For the purpose of this presentation (and reliability) the data is

limited to results from surveys in the financial services industry in

Southern Africa

• Respondent selection for each of the surveys under consideration

was quota-based from client contact lists on proportional stratified

sample designs

• At the time of the interview, the respondent was a current

customer of the financial service provider being evaluated, and

filter-controlled for having a recent interaction at a specific

channel (enterprise-wide metrics across channels across segments)

Slide 34


Research Design & Data Collection

• Survey data was collected via telephonic, web-based and face-to-

face interviews

• To ensure minimal non-sampling error, all interviews were subject

to strict quality assurance processes, and advanced technology was

used to capture data

• No ethical issues are relevant to the study since most of the

findings will be reported at meta-data levels without identifying

any specific sponsoring company (to protect confidentiality and

proprietary measures)

• A strict ESOMAR code-of-conduct was followed in all data

collection. The respondents were made aware of the institutions

sponsoring the survey and for what purposes the information would

be used

Slide 35


Index








• Conclusion

Slide 36


Research Methodology & Instruments

• Prof Adré Schreuder developed a conceptual cause-

and-effect model illustrated as an integrated

customer experience measurement

• Developed through years of academic research

combined with extensive experience regarding

Customer Satisfaction measurement across multiple

industries

• Basis for measurement is a structural model of

customer satisfaction that incorporates the

important constructs of satisfaction that will

identify underlying service or product deficiencies

(or strengths) and a proprietary algorithm for

integrating net measures into this multi-attribute

model

Slide 37


The CONSULTA Integrated Customer Experience

Measurement Model

FAILUREFAILURE DELIGHTDELIGHT




The Conceptual Model Flow

Copyright © Consulta Research - 2010


Principle Calculation of Modeled Scores

FAILUREFAILUREDELIGHTDELIGHT


Instrument Development Process


Model Development Process


Use an Enterprise-wide Model – A Retail Banking

example

Slide 43


Present CE Metrics in Dashboards

Slide 44 Slide 44



Slide 45

For this reason the customer experience index score is not reported in isolation as a single number, but merely as the net

result of multiple items, each of which contains detail results and offers valuable strategic information into the management of

customer delight, loyalty, propensity to shift, service recovery, corrective improvement measures and consequence management

It is important to be able to delve deeper into the results to enable the receiver to delve deeper than satisfaction

The integrated customer experience measurement, although resulting in a final index score, acknowledges the fact that a single

value for an index might hide more that it reveals



• Research Instruments: – Same basic layout including sections corresponding to the

components contained in the conceptual model for customer

satisfaction measurement

– First section measures specific channel’s value proposition with

a range of custom designed service attributes - incorporates

both customer perception and customer expectation by using

confirmation-disconfirmation scale

– Specific questions on product quality, service quality,

relationship quality & pricing as contributing

factors/components of customer satisfaction

Slide 46

0 1 2 4 5 3 9 10 8 7 6

Much worse than expected Much better than expected


Meta-data and Analysis

• For each of the surveys the statistical analysis (using the statistical

software package STATISTICA) included:

– reliability and factor analysis;

– structural equation modelling;

– multiple regression analysis

• The result, for each of the surveys, was a unique structural (cause-

and-effect) model of customer satisfaction that considers all the

important drivers of satisfaction

• Final data set used for meta-analysis contained each of the

components defined on next slide

• Included 704 separate customer satisfaction studies forming part

of the enterprise wide measurement of customer experience, for

each of the financial institutions - each with a sample of at least

100 respondents and more

Slide 47



Slide 48

Metric Description

Weighted service

attribute average score

A weighted average of the (unique channel) service

attributes measured in terms of customer expectation

Service problems % Proportion of respondents who indicated that they

experienced a service problem within a certain time period.

This is different from the proportion of respondents

complaining (formally or informally) as measured in ACSI

Problem recovery % Proportion of respondents who indicated that their service

problem was recovered to their satisfaction

Overall delight % Proportion of respondents who gave a 9 or 10 rating out of

10 for overall satisfaction. This is much more strict than the

typical Top 2 Box metric calculated on a 5 point verbal scale

or the „equivalent‟ “top four” boxes on the ten-point ACSI

scale



Slide 49

Metric Description

Overall failure % Proportion of respondents who gave a 0 or 1 rating out of

10 for overall satisfaction

Average score (overall

satisfaction)

A simple average of overall satisfaction rated on a scale

from 0 to 10

Customer satisfaction

index score

Index score (out of 100) is a function of the following key

elements:

Underlying structural model

Basic calculation principle of being “rewarded” for

positive ratings and being “penalised” for negative

ratings – corresponding to the concept of a net measure

Net Promoter Score Calculated according to the original definition of Reichheld

(2003) the Net Promoter Score equals the % of promoters

minus the % of detractors


Index








• Conclusion

Slide 50


The Dangers of Reporting Net Measures in

Isolation

• Danger/weakness in

reporting any net

measure (in isolation):

two measurements

having exactly the

same value for the net

measure can in fact

have a range of

different values

assigned to the

components of the net

measure

Slide 51


The Dangers of Reporting Net Measures in

Isolation

• Recommendation not only

applicable to net

measures, but to other

“simple” statistical

measures (e.g. the sample

mean) as well

• A variety of different

respondent values can

also yield the same result

for the specific statistical

measure and typical

“distribution” detail

and/or graphs provide

more insight into the

results

Slide 52


Satisfaction Measures as Predictors of the NPS

• As is to be expected, service problems and failure ratings show a

negative correlation with customer satisfaction and NPS, while

delight ratings show a positive correlation. Service problem

recovery shows a very low, but positive, correlation with the NPS –

NOTE poor R2

Slide 53 Sample Base: 1.5million respondents



Slide 54



• Individually, as independent variables in modelling the Customer

Loyalty, the graphs and correlation coefficients clearly show that

the integrated index score with an R2 of 0.73 seems to be the best

predictor of the Net Promoter Score

Slide 55 Sample Base: 1.5million respondents


Satisfaction Measures as Predictor

However, we do not recommend either the NPS or customer satisfaction index score in isolation as “the best and sufficient measurement to evaluate business

performance”, but agree with Schneider et al. that

“using a variety of measures rather than simply one measure would better capture the complexity underlying customer satisfaction and customer

behaviours”

Slide 56 Schneider, D.; Berent, M.; Thomas, R. & Krosnick, J. (2008). Measuring Customer Satisfaction and Loyalty: Improving the Net-Promoter Score. Poster presented at the Annual Meeting of the American Association for Public Opinion Research, New Orleans, Louisiana


Integrated Satisfaction Measure as Predictor

The net measure(s) in itself can provide a top line measurement to track performance or even be effectively used as a “top-of-house” executive

indicator

Analysing the detail of all the different metrics constituting the customer satisfaction index score and NPS will assist greatly in the need for root

cause analyses and strategic/tactical direction

The quantitative data analysis of these measures is further enriched by qualitative questions similar to the “whys” asked by GE, including verbatim descriptions of service problems that were experienced,

suggestions on improving service delivery, etc.

Slide 57


Normality of Customer Experience

Modelled Score

• Due to more complex nature of its calculation, efforts to examine

statistical properties of net measures using a mathematical

approach can be tedious and difficult

• Computer-intensive simulation methods such as the bootstrap

provide a solution

• The bootstrap method was applied to replicate 1 000 bootstrap

samples for each of four different studies – each bootstrap sample

consisted of 380 respondents chosen randomly (with replacement)

from the survey data

• This provided 1 000 simulated index scores, which can be plotted

as histograms and normal probability plots

The accuracy of the simulations increase as the number of bootstrap replications

increase; 500 or more simulations are sufficient to reduce variability and provide

accurate results

Slide 58


Normality of Customer Experience Modelled Score

Slide 59

Variable: VoC1, Distribution: Normal

Chi-Square test = 8.67399, df = 9 (adjusted) , p = 0.46790

50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69

Category (upper limits)

0

2

4

6

8

10

12

14

16

18

20

Re

lative

Fre

qu

en

cy (

%)



40 41 42 43 44 45 46 47 48 49 50 51 52 53 54


0

5

10

15

20

25

Re

lative

Fre

qu

en

cy (

%)



28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43


0

5

10

15

20

25

Re

lative

Fre

qu

en

cy (

%)



32 33 34 35 36 37 38 39 40 41 42 43 44 45 46


0

5

10

15

20

25

Re

lative

Fre

qu

en

cy (

%)

Normal Probability Plot of VoC1 (4 VoCs for normality graphs 4v*1000c)

50 52 54 56 58 60 62 64 66 68

Observed Value

-5

-4

-3

-2

-1

0

1

2

3

4

Exp

ecte

d N

orm

al V

alu

e

VoC1: SW-W = 0.998084051, p = 0.3196


40 42 44 46 48 50 52 54

Observed Value

-4

-3

-2

-1

0

1

2

3

4

Exp

ecte

d N

orm

al V

alu

e

VoC2: SW-W = 0.998708772, p = 0.6945


28 30 32 34 36 38 40 42 44

Observed Value

-4

-3

-2

-1

0

1

2

3

4

Exp

ecte

d N

orm

al V

alu

e

VoC3: SW-W = 0.998033823, p = 0.2971


32 34 36 38 40 42 44 46

Observed Value

-4

-3

-2

-1

0

1

2

3

4

Exp

ecte

d N

orm

al V

alu

e

VoC4: SW-W = 0.998200047, p = 0.3767



Slide 60



50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69


0

2

4

6

8

10

12

14

16

18

20

Re

lative

Fre

qu

en

cy (

%)



40 41 42 43 44 45 46 47 48 49 50 51 52 53 54


0

5

10

15

20

25

Re

lative

Fre

qu

en

cy (

%)



28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43


0

5

10

15

20

25

Re

lative

Fre

qu

en

cy (

%)



32 33 34 35 36 37 38 39 40 41 42 43 44 45 46


0

5

10

15

20

25

Re

lative

Fre

qu

en

cy (

%)


50 52 54 56 58 60 62 64 66 68

Observed Value

-5

-4

-3

-2

-1

0

1

2

3

4

Exp

ecte

d N

orm

al V

alu

e

VoC1: SW-W = 0.998084051, p = 0.3196


40 42 44 46 48 50 52 54

Observed Value

-4

-3

-2

-1

0

1

2

3

4

Exp

ecte

d N

orm

al V

alu

e

VoC2: SW-W = 0.998708772, p = 0.6945


28 30 32 34 36 38 40 42 44

Observed Value

-4

-3

-2

-1

0

1

2

3

4

Exp

ecte

d N

orm

al V

alu

e

VoC3: SW-W = 0.998033823, p = 0.2971


32 34 36 38 40 42 44 46

Observed Value

-4

-3

-2

-1

0

1

2

3

4

Exp

ecte

d N

orm

al V

alu

e

VoC4: SW-W = 0.998200047, p = 0.3767



• For all four studies, both the chi-square test and Shapiro-Wilk test

did NOT reject normality of the customer satisfaction index score,

which holds the benefit of statistical inference of the index score

(e.g. calculating confidence intervals and performing hypothesis

testing)

• Although these results are based on only four studies, representing

a small portion of the wide range of underlying models used to

describe the results of the various studies, we believe that with

additional research we will be able to establish similar results for

the whole range of studies under consideration, and consequently

establish normality for the customer satisfaction index score in

general

Slide 61


Index





• Research Results & Discussion – Reporting Net Measures in Isolation



• Conclusion

Slide 62


Conclusion

• Without denying the fact that net measures has a role to play, the

use of net measures as standalone questions has been shown to

have some disadvantages

• Reporting net measures in context, supported by the multiple

items it contains, provides the opportunity to analyse the detail of

all the different metrics constituting the net measure

• This assist in the need for root cause analyses and

strategic/tactical direction, while the net measure in itself can

provide a top line measurement to track performance or even be

effectively used as a “top-of-house” executive indicator

• The quantitative data analysis of these measures can further be

enriched by qualitative questions, including verbatim descriptions

of service problems that were experienced, suggestions on

improving service delivery, etc.

Slide 63


Conclusion

• Using longitudinal meta-data analysis of more than 1.5

million customer satisfaction measurement interviews,

we have presented reliable correlations between the

Net Promoter Score and an Integrated Customer

Satisfaction Index score, as well as establishing

statistical properties of these measures

• The Customer Satisfaction Index score can be classified

as a combined multi-attribute and net measure

approach, since it incorporates the net effect of

“failure” and “delight” ratings, as well as service

problems and the recovery thereof

Slide 64


Conclusion

Understanding that customers, as human beings, are complex by nature and accepting that the

measurement of customer satisfaction involves the measurement of a complex construct, the use of an

integrated measure of multiple-item & net measures has the advantage of providing insight into

underlying drivers of customer satisfaction, while also offering a simple “top-of-house” dashboard

metric that is simple to communicate.

Slide 65


“A new customer experience measurement model – a meta analytical review of findings over the...

Business

Transcript of “A new customer experience measurement model – a meta analytical review of findings over the...