Download - Performance Measurement and Customer Feedbackservices.byu.edu/emba/course/materials/Course_Slides/...Measuring productivity Performance Measurement and Customer Feedback 9 Dr. Scott

Dr. Scott E. Sampson

Dr. Scott Sampson

James M. Passey Professor of

Service Operations

Brigham Young University

Provo, Utah, USA

Performance

Measurement and

Customer Feedback

rev 3/4/14

Performance Measurement and Customer Feedback 1


Measurement

“What gets measured,

gets managed.”Is this a good

thing or a bad thing?



How do we measure goods quality?



How do we measure service quality?


Ch_CF_instrument_examples


The risk of measuring people




You get what you measure!


But what do we measure?


Measuring productivity



Driver

process domain

interaction Independent processing

Toyota

process domain

Independent processing interaction

Measuring productivity?

produce

car



Driver

process domain


Toyota

process domain


Measuring productivity?

drive car



Driver

process domain


Toyota

process domain


Measuring productivity

produce

carsell car drive car



Provider’s Process Domain Customer’s Process Domain

Independent processing Surrogate interaction Direct interaction Direct interaction Surrogate interaction Independent processing

Measuring Productivity

completed

units

meeting

design

standard

met needs

“realizing value”

meeting needs efficiently

creating “value

potential”






Performance standards?

• recollection of facts

• ability to follow procedures

• ability to make rule-based

decisions

• “knowledge”










decisions

• “knowledge”


Questions:

Are we training students to do things that can

be automated? For example…

• Instead of teaching calculus should we be

teaching people how to effectively use

Wolfram Alpha?

• Instead of teaching accounting skills should

we spend our time writing better accounting

software?

• Instead of even teaching should we be

spending our time developing courseware?





Alternate performance standards?

• ability to deal with decision ambiguity

• critical thinking

• ability to innovate

• social skills and persuasiveness

• ability to work in teams



What gets measured, gets managed

“Teaching”• easy to measure

• easy to automate

“Education”• difficult to measure

• difficult to automate






decisions

• “knowledge”

Alternate performance standards?

• ability to deal with decision ambiguity

• critical thinking

• ability to innovate

• social skills and persuasiveness

• ability to work in teams

Do we measure what is convenient

and quantifiable or what we want?


review CF-instrument-examples

Challenges with

customer

measurement



Six challenges with customer measurement

1. Subjective Rulers

2. Intrusive Measurement

3. Resistance to Measurement

4. The Halo Effect

5. Self-Selected Sampling

6. Interpreting the Interpretations



1. Subjective Rulers

• “Customer measures of quality are generally

subjective.”

• “Making a subjective measure numerical does not

make it objective.”

• How do we analyze this?

• Does a mean make sense?

How would you rate our food?

1. poor

2. ok

3. good

4. very good

5. exceptional



2. Intrusive Measurement

• “The act of customer-measurement of quality can

influence perceptions.”

• Hawthorn effect.

1. Was there anything about

your stay that we could have

improved?

2. How would rate our service?



3. Resistance to Measurement

• “Most customers do not consider quality measurement to

be value adding, therefore resist providing

measurements.”

• “This resistance increases as the customers’ cost of

providing measurement increases.”



4. The Halo Effect

• “Customers automatically combine individual components

of quality into an overall quality perception.”

• “Attempted measurements of individual components may

actually have more to do with the overall perception than

the individual components.”

How would you rate his…

1.leadership?

2.intelligence?

3.diplomacy?

4.wit?

5.singing voice?



5. Self-Selected Sampling• “Customer-measurement makes it is possible to influence

sampling, but very difficult to control sampling.”

• “When we control sampling we know how survey-responding customers (sampled) compare with customers in general.”

“Strategies to increase

response rates, such

as awards or drawing

for prizes, influence

some types of

customers more than

others.”

“Therefore, it is

important to consider

sample bias.”



5. Self-Selected Sampling

• Sample bias



6. Interpreting the Interpretations • “Customer measurement requires the customer to interpret both their perceptions and the measurement scale.”

• “Two customers with identical perceptions might interpret the measurement scale differently, resulting in different measurements.”

How did we do?

☐poor

☐ok

☐good

☐excellent

How did we do?

☐poor

☐ok

☐good

☐excellent



Performance Measurement and Customer Feedback 27juncture


Customer

measurement ROI



Typical customer measurement systems

• “…in a study of 22 large ‘customer driven’

companies, [TARP] found that the companies

spent a median of $1 million and employed the

equivalent of 13 full-time professional staff per

year on customer feedback systems.”

“TARP researchers report that

‘many of these companies

have little to show for the

investment’”

Why not?source: Sampson, S. E. 1999, “An Empirically Defined Framework for Designing Customer

Feedback Systems,” Quality Management Journal, Vol. 6, No. 3, pp. 64-80.



An inconvenient truth…

Quality customer

feedback requires an

investment, therefore

should produce a return.



Developing an

effective customer

feedback system



Designing effective customer feedback systems

• Step 1…

Begin with

the end in

mind!

Plan your work

then work your

plan!




• Where?

• Why?

• What?

• How?

• When?

• Who?




• Where?

• Why?

• What?

• How?

• When?

• Who?

Functional target?

• design

• marketing

• operations

• customer support

Feedback goal?

(see CFSD

tables)


Service-

Product

Design

Goals

Marketing

Goals

Operations

Goals

Customer

Support

Goals

Ascertain

Customer

Needs

Identify

Customers

Measure

Specification

Conformance

Evaluate

Customer

Satisfaction

Develop

Service

Product

Promote

Service

Product

Compare

Across

Organization

Assure

Repeat

Business

Benchmark

Versus

Competition

Generate

Sales

Leads

Increase

Employee

Involvement

Provide

Sounding

Board



• Where?

• Why?

• What?

• How?

• When?

• Who?

Functional target?

• design

• marketing

• operations


Feedback goal?

(see CFSD

tables)

Objective frame?

• short-term

• long-term

Questioning?

Response type?

• dichotomous

• scale

• list (to check)

• field (fill-in-blank)

• open-ended

Data use?

• quantitative

• qualitative

Bias sensitive?

yes ↔ no

Solicitation type?

active ↔ passive

Extent of questioning?

brief ↔ extensive

Feedback incentive? Visibility?

Temporal frame?

• pre-process

• in process

• post-process

Channeling?

• who collects?

• who reviews?

• who responds?

• who follows up?



Summary

• No royal road to effective service measurement.

• Understand challenges of measuring customers.

• Take a systematic approach.

• Begin with the end in mind!



Performance Measurement and Customer Feedback 38end


Sample goals of customer feedback


Service-Product

Design Goals

Marketing

Goals

Operations

Goals

Customer

Support Goals

Ascertain

Customer Needs

Identify

Customers

Measure

Specification

Conformance

Evaluate

Customer

Satisfaction

Develop Service

Product

Promote Service

Product

Compare

Across

Organization

Assure

Repeat

Business

Benchmark

Versus

Competition

Generate

Sales

Leads

Increase

Employee

Involvement

Provide

Sounding Board


CF Goals relating to Service-Product Design


Goal(S)hort-term and (L)ong-term

Objectives

Survey Question Examples

(showing response types*)

How to Use the Feedback

(sensitive to response bias?)

Ascertain Customer

Needs

(S) Assess: Determine the

strength of various customer

needs being met.

How important are each of the

following product features... (S)

Means and histograms. Use to

help prioritize service product

improvement efforts. (yes)

(L) Probe: Determine important

needs potentially addressed by

the service product.

What features are important for

this type of service product? (L

or F) What other needs we

might serve? (F, O)

Use as a discussion topic for

focus groups with customer

groups. (no)

Develop Service

Product

(S) Check: Assure that the

service product is meeting the

objectives it was designed to

meet.

How would you rate us

according to the following

objectives... (S)

Means and histograms.

Identify service product

weaknesses or fail points. (yes)

(L) Prescribe: Identify ideas for

future features or service

products.

Suggestions? Possible

improvements? (O)

Use as seed ideas in the design

process. (no)

Benchmark Versus

Competition

(S) Compare: Determine how

our company is viewed relative

to the competition.

How would you rate us relative

to the competition... (S)

Means and histograms. Can

help identify where we need to

study the competition. (no)

(L) Borrow: Determine ways in

which the competition provides

a superior service product.

What do you particularly like

about our competitors? (O)

Use as guidance for studying

competitors’ service products.

(no)

*Question response types: Dichotomous (e.g., yes/no), Scale (e.g., 1 to 7), List to check, Field (Name/address/phone), Open ended, Any.


CF Goals relating to Marketing



Objectives





Identify Customers

(S) Describe: Determine what

type of people our customers

are.

Age? Income? (other

demographic questions) (L, F)

Means and histograms. Assist

in targeting and advertising.

(yes)

(L) Trace: Discover why/how

our customers came to be our

customers.

How did you select us? What

features were major selling

points? (L or O)

Analyze effectiveness of

various advertising and sales

efforts. (yes)

Promote Service

Product

(S) Impress: Give the

perception that the company

cares about customers and

quality.

(any questions about perceived

quality and meeting customer

needs) (A)

Respond to feedback, when

possible (i.e. have name and

address or phone). (no)

(L) Publicize: Collect

exceptional service product

reviews for future publicizing.

Comments about our

company? Anything you

consider exceptional? (O)

Publish excerpts in company

and other publications. (no)

Generate

Sales

Leads

(S) Push: Invite customers to

repurchase.

Which of these (service

products) can we send you

more information about? (L, N)

Forward to sales department.

(no)

(L) Enroll: Encourage

customers to join a frequent

purchase club or mailing list.

Would you like to join our

(membership club or mailing

list)? (N)

Forward to sales department.

(no)



CF Goals relating to Operations



Objectives





Measure Specification

Conformance

(S) Inspect: Determine if the

service product performed to

formal specifications.

Did (such-and-such service

product feature) occur? (D)

Feed failure information back to

employees responsible for

providing that feature. (no)

(L) Track: Track areas of

potential variance to assure the

process remains in control over

time.

Rate the following features we

provide... (S)

Use standard control-chart

analysis to identify if and when

the process goes out of control.

(yes)

Compare

Across

Organization

(S) Focus: Help focus

management on company sites

or departments in need of

attention.

What department/location did

you visit? Comments? (look for

problems) How do they rate...?

(F, O, S)

Summarize data by

department/ location, looking

for patterns, outliers, and high

variance. (no)

(L) Baseline: Develop

performance standards based

on best practices occurring in

the organization.

What department/location did

you visit? (L) How did they do

in these areas... (F, S)

Summarize data and distribute

to each department/location,

highlighting top results. (yes)

Increase Employee

Involvement

(S) Motivate: Provide the basis

for rewarding or punishing

employees for the service

product they provide.

Ratings for... (employees or

their work). (S) Employee to

nominate for recognition?

Why? (F, O)

Recognize or reward employ-

ees for top ratings or significant

nomination by customers. (no)

(L) Empower: Direct

employees in development of

their area of responsibility.

How might we improve...

(specific aspects of the service

product) (O)

Direct the feedback to

employees, who then report

how it is handled. (no)



CF Goals relating to Customer Support



Objectives





Evaluate Customer

Satisfaction

(S) Reassure: Receive

assurance that customers are

generally satisfied.

Did we meet your expectations?

(D) How do we rate overall? (S)

Means and histograms. Com-

pare with target rating values.

(yes)

(L) Protect: Watch for customer

problems that could potentially

become serious (such as legal

liability).

Complaints? Comments?

(focusing on complaint

responses) (O)

Investigate complaints, espec-

ially reoccurring. Resolve prob-

lems before they turn serious.

(no)

Assure

Repeat

Business

(S) Recover: Attempt to

reconcile and reclaim

dissatisfied customers.

Any problems with...? Were you

satisfied with...? (D, N)

Contact customers when

possible and offer apology and

remuneration. (no)

(L) Retain: Determine reasons

customers defect to prevent

future defections.

What might cause you to

choose the competition? (O)

How likely is that to occur? (S)

Build checks into the service

product to assure defection-

inducing problems are avoided.

(no)

Provide Sounding

Board

(S) Hear: Allow customers the

opportunity to speak their

minds.

Comments? (O) When possible, thank customer

for feedback. Apply other

objectives as appropriate. (no)

(L) Communicate: Provide the

opportunity to interact with

customers.

Questions or comments? (O)

Would you like a response? (D)

Name and address? (N)

Tell the customer how you acted

on the feedback, and possibly

ask for more feedback. (no)



Practice ExerciseBest way to collect student feedback?



Best way to collect student feedback?

1. Determine how to improve courses?

2. Help promote courses to future students?

3. Identify the best instructors?

4. Measure student satisfaction?

5. Motivate faculty to be more responsive?

6. Uncover how to make courses more engaging?

7. Help course meet more student needs?


Service-Product Design Goals Marketing Goals Operations Goals Customer Support Goals

Ascertain Customer Needs Identify Customers Measure Specification Conformance Evaluate Customer Satisfaction

Develop Service Product Promote Service Product Compare Across Organization Assure Repeat Business

Benchmark Versus Competition Generate Sales Leads Increase Employee Involvement Provide Sounding Board

Develop Service Product

Promote Service Product

Compare Across Organization

Evaluate Customer Satisfaction

Increase Employee Involvement

Benchmark Versus Competition

Ascertain Customer Needs


Best way to collect student feedback?

1. Determine how to improve courses?

2. Help promote courses to future students?

3. Identify the best instructors?

4. Measure student satisfaction?

5. Motivate faculty to be more responsive?

6. Uncover how to make courses more engaging?

7. Help course meet more student needs?


Service-Product Design Goals Marketing Goals Operations Goals Customer Support Goals

Ascertain Customer Needs Identify Customers Measure Specification Conformance Evaluate Customer Satisfaction

Develop Service Product Promote Service Product Compare Across Organization Assure Repeat Business

Benchmark Versus Competition Generate Sales Leads Increase Employee Involvement Provide Sounding Board