Performance Measurement and Customer...

of 46/46
Dr. Scott E. Sampson Dr. Scott Sampson James M. Passey Professor of Service Operations Brigham Young University Provo, Utah, USA Performance Measurement and Customer Feedback rev 3/4/14 Performance Measurement and Customer Feedback 1
  • date post

    16-Jul-2020
  • Category

    Documents

  • view

    2
  • download

    0

Embed Size (px)

Transcript of Performance Measurement and Customer...

  • Dr. Scott E. Sampson

    Dr. Scott Sampson

    James M. Passey Professor of

    Service Operations

    Brigham Young University

    Provo, Utah, USA

    Performance

    Measurement and

    Customer Feedback

    rev 3/4/14

    Performance Measurement and Customer Feedback 1

  • Dr. Scott E. Sampson

    Measurement

    “What gets measured,

    gets managed.”Is this a good

    thing or a bad thing?

    Performance Measurement and Customer Feedback 2

  • Dr. Scott E. Sampson

    How do we measure goods quality?

    Performance Measurement and Customer Feedback 3

  • Dr. Scott E. Sampson

    How do we measure service quality?

    Performance Measurement and Customer Feedback 4

    Ch_CF_instrument_examples

  • Dr. Scott E. Sampson

    The risk of measuring people

    Performance Measurement and Customer Feedback 5

  • Dr. Scott E. Sampson

    The risk of measuring people

    Performance Measurement and Customer Feedback 6

  • Dr. Scott E. Sampson

    The risk of measuring people

    Performance Measurement and Customer Feedback 7

  • Dr. Scott E. Sampson

    The risk of measuring people

    You get what you measure!

    Performance Measurement and Customer Feedback 8

    But what do we measure?

  • Dr. Scott E. Sampson

    Measuring productivity

    Performance Measurement and Customer Feedback 9

  • Dr. Scott E. Sampson

    Driver

    process domain

    interaction Independent processing

    Toyota

    process domain

    Independent processing interaction

    Measuring productivity?

    produce

    car

    Performance Measurement and Customer Feedback 10

  • Dr. Scott E. Sampson

    Driver

    process domain

    interaction Independent processing

    Toyota

    process domain

    Independent processing interaction

    Measuring productivity?

    drive car

    Performance Measurement and Customer Feedback 11

  • Dr. Scott E. Sampson

    Driver

    process domain

    interaction Independent processing

    Toyota

    process domain

    Independent processing interaction

    Measuring productivity

    produce

    carsell car drive car

    Performance Measurement and Customer Feedback 12

  • Dr. Scott E. Sampson

    Provider’s Process Domain Customer’s Process Domain

    Independent processing Surrogate interaction Direct interaction Direct interaction Surrogate interaction Independent processing

    Measuring Productivity

    completed

    units

    meeting

    design

    standard

    met needs

    “realizing value”

    meeting needs efficiently

    creating “value

    potential”

    Performance Measurement and Customer Feedback 13

  • Dr. Scott E. Sampson

    Provider’s Process Domain Customer’s Process Domain

    Independent processing Surrogate interaction Direct interaction Direct interaction Surrogate interaction Independent processing

    Measuring Productivity

    Performance standards?

    • recollection of facts

    • ability to follow procedures

    • ability to make rule-based

    decisions

    • “knowledge”

    Performance Measurement and Customer Feedback 14

  • Dr. Scott E. Sampson

    Provider’s Process Domain Customer’s Process Domain

    Independent processing Surrogate interaction Direct interaction Direct interaction Surrogate interaction Independent processing

    Measuring Productivity

    Performance standards?

    • recollection of facts

    • ability to follow procedures

    • ability to make rule-based

    decisions

    • “knowledge”

    Performance Measurement and Customer Feedback 15

    Questions:

    Are we training students to do things that can

    be automated? For example…

    • Instead of teaching calculus should we be

    teaching people how to effectively use

    Wolfram Alpha?

    • Instead of teaching accounting skills should

    we spend our time writing better accounting

    software?

    • Instead of even teaching should we be

    spending our time developing courseware?

  • Dr. Scott E. Sampson

    Provider’s Process Domain Customer’s Process Domain

    Independent processing Surrogate interaction Direct interaction Direct interaction Surrogate interaction Independent processing

    Measuring Productivity

    Alternate performance standards?

    • ability to deal with decision ambiguity

    • critical thinking

    • ability to innovate

    • social skills and persuasiveness

    • ability to work in teams

    Performance Measurement and Customer Feedback 16

  • Dr. Scott E. Sampson

    What gets measured, gets managed

    “Teaching”• easy to measure

    • easy to automate

    “Education”• difficult to measure

    • difficult to automate

    Performance Measurement and Customer Feedback 17

    Performance standards?

    • recollection of facts

    • ability to follow procedures

    • ability to make rule-based

    decisions

    • “knowledge”

    Alternate performance standards?

    • ability to deal with decision ambiguity

    • critical thinking

    • ability to innovate

    • social skills and persuasiveness

    • ability to work in teams

    Do we measure what is convenient

    and quantifiable or what we want?

  • Dr. Scott E. Sampson

    review CF-instrument-examples

    Challenges with

    customer

    measurement

    Performance Measurement and Customer Feedback 18

  • Dr. Scott E. Sampson

    Six challenges with customer measurement

    1. Subjective Rulers

    2. Intrusive Measurement

    3. Resistance to Measurement

    4. The Halo Effect

    5. Self-Selected Sampling

    6. Interpreting the Interpretations

    Performance Measurement and Customer Feedback 19

  • Dr. Scott E. Sampson

    1. Subjective Rulers

    • “Customer measures of quality are generally

    subjective.”

    • “Making a subjective measure numerical does not

    make it objective.”

    • How do we analyze this?

    • Does a mean make sense?

    How would you rate our food?

    1. poor

    2. ok

    3. good

    4. very good

    5. exceptional

    Performance Measurement and Customer Feedback 20

  • Dr. Scott E. Sampson

    2. Intrusive Measurement

    • “The act of customer-measurement of quality can

    influence perceptions.”

    • Hawthorn effect.

    1. Was there anything about

    your stay that we could have

    improved?

    2. How would rate our service?

    Performance Measurement and Customer Feedback 21

  • Dr. Scott E. Sampson

    3. Resistance to Measurement

    • “Most customers do not consider quality measurement to

    be value adding, therefore resist providing

    measurements.”

    • “This resistance increases as the customers’ cost of

    providing measurement increases.”

    Performance Measurement and Customer Feedback 22

  • Dr. Scott E. Sampson

    4. The Halo Effect

    • “Customers automatically combine individual components

    of quality into an overall quality perception.”

    • “Attempted measurements of individual components may

    actually have more to do with the overall perception than

    the individual components.”

    How would you rate his…

    1.leadership?

    2.intelligence?

    3.diplomacy?

    4.wit?

    5.singing voice?

    Performance Measurement and Customer Feedback 23

  • Dr. Scott E. Sampson

    5. Self-Selected Sampling• “Customer-measurement makes it is possible to influence

    sampling, but very difficult to control sampling.”

    • “When we control sampling we know how survey-responding customers (sampled) compare with customers in general.”

    “Strategies to increase

    response rates, such

    as awards or drawing

    for prizes, influence

    some types of

    customers more than

    others.”

    “Therefore, it is

    important to consider

    sample bias.”

    Performance Measurement and Customer Feedback 24

  • Dr. Scott E. Sampson

    5. Self-Selected Sampling

    • Sample bias

    Performance Measurement and Customer Feedback 25

  • Dr. Scott E. Sampson

    6. Interpreting the Interpretations • “Customer measurement requires the customer to interpret both their perceptions and the measurement scale.”

    • “Two customers with identical perceptions might interpret the measurement scale differently, resulting in different measurements.”

    How did we do?

    ☐poor☐ok☐good☐excellent

    How did we do?

    ☐poor☐ok☐good☐excellent

    Performance Measurement and Customer Feedback 26

  • Dr. Scott E. Sampson

    Performance Measurement and Customer Feedback 27juncture

  • Dr. Scott E. Sampson

    Customer

    measurement ROI

    Performance Measurement and Customer Feedback 28

  • Dr. Scott E. Sampson

    Typical customer measurement systems

    • “…in a study of 22 large ‘customer driven’

    companies, [TARP] found that the companies

    spent a median of $1 million and employed the

    equivalent of 13 full-time professional staff per

    year on customer feedback systems.”

    “TARP researchers report that

    ‘many of these companies

    have little to show for the

    investment’”

    Why not?source: Sampson, S. E. 1999, “An Empirically Defined Framework for Designing Customer

    Feedback Systems,” Quality Management Journal, Vol. 6, No. 3, pp. 64-80.

    Performance Measurement and Customer Feedback 29

  • Dr. Scott E. Sampson

    An inconvenient truth…

    Quality customer

    feedback requires an

    investment, therefore

    should produce a return.

    Performance Measurement and Customer Feedback 30

  • Dr. Scott E. Sampson

    Developing an

    effective customer

    feedback system

    Performance Measurement and Customer Feedback 31

  • Dr. Scott E. Sampson

    Designing effective customer feedback systems

    • Step 1…

    Begin with

    the end in

    mind!

    Plan your work

    then work your

    plan!

    Performance Measurement and Customer Feedback 32

  • Dr. Scott E. Sampson

    Designing effective customer feedback systems

    • Where?

    • Why?

    • What?

    • How?

    • When?

    • Who?

    Performance Measurement and Customer Feedback 33

  • Dr. Scott E. Sampson

    Designing effective customer feedback systems

    • Where?

    • Why?

    • What?

    • How?

    • When?

    • Who?

    Functional target?

    • design

    • marketing

    • operations

    • customer support

    Feedback goal?

    (see CFSD

    tables)

    Performance Measurement and Customer Feedback 34

    Service-

    Product

    Design

    Goals

    Marketing

    Goals

    Operations

    Goals

    Customer

    Support

    Goals

    Ascertain

    Customer

    Needs

    Identify

    Customers

    Measure

    Specification

    Conformance

    Evaluate

    Customer

    Satisfaction

    Develop

    Service

    Product

    Promote

    Service

    Product

    Compare

    Across

    Organization

    Assure

    Repeat

    Business

    Benchmark

    Versus

    Competition

    Generate

    Sales

    Leads

    Increase

    Employee

    Involvement

    Provide

    Sounding

    Board

  • Dr. Scott E. Sampson

    Designing effective customer feedback systems

    • Where?

    • Why?

    • What?

    • How?

    • When?

    • Who?

    Functional target?

    • design

    • marketing

    • operations

    • customer support

    Feedback goal?

    (see CFSD

    tables)

    Objective frame?

    • short-term

    • long-term

    Questioning?

    Response type?

    • dichotomous

    • scale

    • list (to check)

    • field (fill-in-blank)

    • open-ended

    Data use?

    • quantitative

    • qualitative

    Bias sensitive?

    yes ↔ no

    Solicitation type?

    active ↔ passive

    Extent of questioning?

    brief ↔ extensive

    Feedback incentive? Visibility?

    Temporal frame?

    • pre-process

    • in process

    • post-process

    Channeling?

    • who collects?

    • who reviews?

    • who responds?

    • who follows up?

    Performance Measurement and Customer Feedback 35

  • Dr. Scott E. Sampson

    Designing effective customer feedback systems

    • Where?

    • Why?

    • What?

    • How?

    • When?

    • Who?

    Functional target?

    • design

    • marketing

    • operations

    • customer support

    Feedback goal?

    (see CFSD

    tables)

    Objective frame?

    • short-term

    • long-term

    Questioning?

    Response type?

    • dichotomous

    • scale

    • list (to check)

    • field (fill-in-blank)

    • open-ended

    Data use?

    • quantitative

    • qualitative

    Bias sensitive?

    yes ↔ no

    Solicitation type?

    active ↔ passive

    Extent of questioning?

    brief ↔ extensive

    Feedback incentive? Visibility?

    Temporal frame?

    • pre-process

    • in process

    • post-process

    Channeling?

    • who collects?

    • who reviews?

    • who responds?

    • who follows up?

    Performance Measurement and Customer Feedback 36

  • Dr. Scott E. Sampson

    Summary

    • No royal road to effective service measurement.

    • Understand challenges of measuring customers.

    • Take a systematic approach.

    • Begin with the end in mind!

    Performance Measurement and Customer Feedback 37

  • Dr. Scott E. Sampson

    Performance Measurement and Customer Feedback 38end

  • Dr. Scott E. Sampson

    Sample goals of customer feedback

    Performance Measurement and Customer Feedback 39

    Service-Product

    Design Goals

    Marketing

    Goals

    Operations

    Goals

    Customer

    Support Goals

    Ascertain

    Customer Needs

    Identify

    Customers

    Measure

    Specification

    Conformance

    Evaluate

    Customer

    Satisfaction

    Develop Service

    Product

    Promote Service

    Product

    Compare

    Across

    Organization

    Assure

    Repeat

    Business

    Benchmark

    Versus

    Competition

    Generate

    Sales

    Leads

    Increase

    Employee

    Involvement

    Provide

    Sounding Board

  • Dr. Scott E. Sampson

    CF Goals relating to Service-Product Design

    Performance Measurement and Customer Feedback 40

    Goal(S)hort-term and (L)ong-term

    Objectives

    Survey Question Examples

    (showing response types*)

    How to Use the Feedback

    (sensitive to response bias?)

    Ascertain Customer

    Needs

    (S) Assess: Determine the

    strength of various customer

    needs being met.

    How important are each of the

    following product features... (S)

    Means and histograms. Use to

    help prioritize service product

    improvement efforts. (yes)

    (L) Probe: Determine important

    needs potentially addressed by

    the service product.

    What features are important for

    this type of service product? (L

    or F) What other needs we

    might serve? (F, O)

    Use as a discussion topic for

    focus groups with customer

    groups. (no)

    Develop Service

    Product

    (S) Check: Assure that the

    service product is meeting the

    objectives it was designed to

    meet.

    How would you rate us

    according to the following

    objectives... (S)

    Means and histograms.

    Identify service product

    weaknesses or fail points. (yes)

    (L) Prescribe: Identify ideas for

    future features or service

    products.

    Suggestions? Possible

    improvements? (O)

    Use as seed ideas in the design

    process. (no)

    Benchmark Versus

    Competition

    (S) Compare: Determine how

    our company is viewed relative

    to the competition.

    How would you rate us relative

    to the competition... (S)

    Means and histograms. Can

    help identify where we need to

    study the competition. (no)

    (L) Borrow: Determine ways in

    which the competition provides

    a superior service product.

    What do you particularly like

    about our competitors? (O)

    Use as guidance for studying

    competitors’ service products.

    (no)

    *Question response types: Dichotomous (e.g., yes/no), Scale (e.g., 1 to 7), List to check, Field (Name/address/phone), Open ended, Any.

  • Dr. Scott E. Sampson

    CF Goals relating to Marketing

    Performance Measurement and Customer Feedback 41

    Goal(S)hort-term and (L)ong-term

    Objectives

    Survey Question Examples

    (showing response types*)

    How to Use the Feedback

    (sensitive to response bias?)

    Identify Customers

    (S) Describe: Determine what

    type of people our customers

    are.

    Age? Income? (other

    demographic questions) (L, F)

    Means and histograms. Assist

    in targeting and advertising.

    (yes)

    (L) Trace: Discover why/how

    our customers came to be our

    customers.

    How did you select us? What

    features were major selling

    points? (L or O)

    Analyze effectiveness of

    various advertising and sales

    efforts. (yes)

    Promote Service

    Product

    (S) Impress: Give the

    perception that the company

    cares about customers and

    quality.

    (any questions about perceived

    quality and meeting customer

    needs) (A)

    Respond to feedback, when

    possible (i.e. have name and

    address or phone). (no)

    (L) Publicize: Collect

    exceptional service product

    reviews for future publicizing.

    Comments about our

    company? Anything you

    consider exceptional? (O)

    Publish excerpts in company

    and other publications. (no)

    Generate

    Sales

    Leads

    (S) Push: Invite customers to

    repurchase.

    Which of these (service

    products) can we send you

    more information about? (L, N)

    Forward to sales department.

    (no)

    (L) Enroll: Encourage

    customers to join a frequent

    purchase club or mailing list.

    Would you like to join our

    (membership club or mailing

    list)? (N)

    Forward to sales department.

    (no)

    *Question response types: Dichotomous (e.g., yes/no), Scale (e.g., 1 to 7), List to check, Field (Name/address/phone), Open ended, Any.

  • Dr. Scott E. Sampson

    CF Goals relating to Operations

    Performance Measurement and Customer Feedback 42

    Goal(S)hort-term and (L)ong-term

    Objectives

    Survey Question Examples

    (showing response types*)

    How to Use the Feedback

    (sensitive to response bias?)

    Measure Specification

    Conformance

    (S) Inspect: Determine if the

    service product performed to

    formal specifications.

    Did (such-and-such service

    product feature) occur? (D)

    Feed failure information back to

    employees responsible for

    providing that feature. (no)

    (L) Track: Track areas of

    potential variance to assure the

    process remains in control over

    time.

    Rate the following features we

    provide... (S)

    Use standard control-chart

    analysis to identify if and when

    the process goes out of control.

    (yes)

    Compare

    Across

    Organization

    (S) Focus: Help focus

    management on company sites

    or departments in need of

    attention.

    What department/location did

    you visit? Comments? (look for

    problems) How do they rate...?

    (F, O, S)

    Summarize data by

    department/ location, looking

    for patterns, outliers, and high

    variance. (no)

    (L) Baseline: Develop

    performance standards based

    on best practices occurring in

    the organization.

    What department/location did

    you visit? (L) How did they do

    in these areas... (F, S)

    Summarize data and distribute

    to each department/location,

    highlighting top results. (yes)

    Increase Employee

    Involvement

    (S) Motivate: Provide the basis

    for rewarding or punishing

    employees for the service

    product they provide.

    Ratings for... (employees or

    their work). (S) Employee to

    nominate for recognition?

    Why? (F, O)

    Recognize or reward employ-

    ees for top ratings or significant

    nomination by customers. (no)

    (L) Empower: Direct

    employees in development of

    their area of responsibility.

    How might we improve...

    (specific aspects of the service

    product) (O)

    Direct the feedback to

    employees, who then report

    how it is handled. (no)

    *Question response types: Dichotomous (e.g., yes/no), Scale (e.g., 1 to 7), List to check, Field (Name/address/phone), Open ended, Any.

  • Dr. Scott E. Sampson

    CF Goals relating to Customer Support

    Performance Measurement and Customer Feedback 43

    Goal(S)hort-term and (L)ong-term

    Objectives

    Survey Question Examples

    (showing response types*)

    How to Use the Feedback

    (sensitive to response bias?)

    Evaluate Customer

    Satisfaction

    (S) Reassure: Receive

    assurance that customers are

    generally satisfied.

    Did we meet your expectations?

    (D) How do we rate overall? (S)

    Means and histograms. Com-

    pare with target rating values.

    (yes)

    (L) Protect: Watch for customer

    problems that could potentially

    become serious (such as legal

    liability).

    Complaints? Comments?

    (focusing on complaint

    responses) (O)

    Investigate complaints, espec-

    ially reoccurring. Resolve prob-

    lems before they turn serious.

    (no)

    Assure

    Repeat

    Business

    (S) Recover: Attempt to

    reconcile and reclaim

    dissatisfied customers.

    Any problems with...? Were you

    satisfied with...? (D, N)

    Contact customers when

    possible and offer apology and

    remuneration. (no)

    (L) Retain: Determine reasons

    customers defect to prevent

    future defections.

    What might cause you to

    choose the competition? (O)

    How likely is that to occur? (S)

    Build checks into the service

    product to assure defection-

    inducing problems are avoided.

    (no)

    Provide Sounding

    Board

    (S) Hear: Allow customers the

    opportunity to speak their

    minds.

    Comments? (O) When possible, thank customer

    for feedback. Apply other

    objectives as appropriate. (no)

    (L) Communicate: Provide the

    opportunity to interact with

    customers.

    Questions or comments? (O)

    Would you like a response? (D)

    Name and address? (N)

    Tell the customer how you acted

    on the feedback, and possibly

    ask for more feedback. (no)

    *Question response types: Dichotomous (e.g., yes/no), Scale (e.g., 1 to 7), List to check, Field (Name/address/phone), Open ended, Any.

  • Dr. Scott E. Sampson

    Practice ExerciseBest way to collect student feedback?

    Performance Measurement and Customer Feedback 44

  • Dr. Scott E. Sampson

    Best way to collect student feedback?

    1. Determine how to improve courses?

    2. Help promote courses to future students?

    3. Identify the best instructors?

    4. Measure student satisfaction?

    5. Motivate faculty to be more responsive?

    6. Uncover how to make courses more engaging?

    7. Help course meet more student needs?

    Performance Measurement and Customer Feedback 45

    Service-Product Design Goals Marketing Goals Operations Goals Customer Support Goals

    Ascertain Customer Needs Identify Customers Measure Specification Conformance Evaluate Customer Satisfaction

    Develop Service Product Promote Service Product Compare Across Organization Assure Repeat Business

    Benchmark Versus Competition Generate Sales Leads Increase Employee Involvement Provide Sounding Board

    Develop Service Product

    Promote Service Product

    Compare Across Organization

    Evaluate Customer Satisfaction

    Increase Employee Involvement

    Benchmark Versus Competition

    Ascertain Customer Needs

  • Dr. Scott E. Sampson

    Best way to collect student feedback?

    1. Determine how to improve courses?

    2. Help promote courses to future students?

    3. Identify the best instructors?

    4. Measure student satisfaction?

    5. Motivate faculty to be more responsive?

    6. Uncover how to make courses more engaging?

    7. Help course meet more student needs?

    Performance Measurement and Customer Feedback 46

    Service-Product Design Goals Marketing Goals Operations Goals Customer Support Goals

    Ascertain Customer Needs Identify Customers Measure Specification Conformance Evaluate Customer Satisfaction

    Develop Service Product Promote Service Product Compare Across Organization Assure Repeat Business

    Benchmark Versus Competition Generate Sales Leads Increase Employee Involvement Provide Sounding Board