Lllll Llllo Ooooooo

download Lllll Llllo Ooooooo

of 176

Transcript of Lllll Llllo Ooooooo

  • 7/26/2019 Lllll Llllo Ooooooo

    1/176

    buys the cereal (usually a parent) or the person that consumes it (often a child)? And what of a complicated buyingdecision in a business to business situation. Who should be interviewed in a customer satisfaction survey for a truckmanufacturer the driver, the transport manager, the general management of the company? n other business tobusiness markets there may well be influences on the buying decision from engineering, production, purchasing,!uality assurance, plus research and development. "ecause each department evaluates suppliers differently, thecustomer satisfaction programme will need to cover the multiple views.

    #he adage in market research that we turn to again and again is the need to ask the right !uestion of the right person.$inding that person in customer satisfaction research may re!uire a compromise with a focus on one person % the keydecision maker& perhaps the transport manager in the e'ample of the trucks. f money and time permit, differentpeople could be interviewed and this may involve different interviewing methods and different !uestions.

    #he traditional first in line customer is an obvious candidate for measuring customer satisfaction. "ut what aboutother people in the channel to market? f the products are sold through intermediaries, we are even further from ourcustomers. A good customer satisfaction program will include at least the most important of these types of channelcustomers, perhaps the wholesalers as well as the final consumers.

    ne of the greatest headaches in the organisation of a business to business customer satisfaction survey is thecompilation of the sample frame the list from which the sample of respondents is selected. "uilding an accurate,up%to%date list of customers, with telephone numbers and contact details is nearly always a challenge. #he list held

    by the accounts department may not have the contact details of the people making the purchasing decision. argebusinesses may have regionally autonomous units and there may be some fiefdom that says it doesn*t want itscustomers pestered by market researchers. #he sales teams* +hristmas card lists may well be the best lists of all butthey are kept close to the chest of each sales person and not held on a central server. "uilding a good sample framenearly always takes longer than was planned but it is the foundation of a good customer satisfaction survey.

    +ustomer satisfaction surveys are often ust that surveys of customers without consideration of the views of lost orpotential customers. apsed customers may have stories to tell about service issues while potential customers are agood source of benchmark data on the competition. f a survey is to embrace non%customers, the compilation of thesample frame is even more difficult. #he !uality of these sample frames influences the results more than any otherfactor since they are usually outside the researchers* control. #he !uestionnaire design and interpretation are withinthe control of the researchers and these are subects where they will have considerable e'perience.

  • 7/26/2019 Lllll Llllo Ooooooo

    2/176

    W-A# -/0 "1 21A/310?

    n customer satisfaction research we seek the views of respondents on a variety of issues that will show how thecompany is performing and how it can improve. #his understanding is obtained at a high level (4how satisfied areyou the A"+ td overall?5) and at a very specific level (4how satisfied are you with the clarity of invoices?5).

    -igh level issues are included in most customer satisfaction surveys and they could be captured by !uestions suchas6

    What is your overall satisfaction with A"+ td?-ow likely or unlikely are you to buy from A"+ td again?-ow likely or unlikely would you be to recommend A"+ td to a friend or colleague?

    t is at the more specific level of !uestioning that things become more difficult. ome issues are of obviousimportance and every supplier is e'pected to perform to a minimum acceptable level on them. #hese are thehygiene factors. f a company fails on any of these issues they would !uickly lose market share or go out ofbusiness. An airline must offer safety but the level of in%flight service is a variable. #hese variables such as in%flightservice are often the issues that differentiate companies and create the satisfaction or dissatisfaction.

    Working out what !uestions to ask at a detailed level means seeing the world from the customers* points of view.

    What do they consider important? #hese factors or attributes will differ from company to company and there couldbe a long list. #hey could include the following6

    About delivery

    0elivery on timepeed of delivery

    #he list is not e'haustive by any means. #here is no mention above of environmental issues, sales literature,fre!uency of representatives* calls or packaging. 1ven though the attributes are deemed specific, it is not entirelyclear what is meant by 4product !uality5 or 4ease of doing business5. +ryptic labels that summarise specific issueshave to be carefully chosen for otherwise it will be impossible to interpret the results.

    +ustomer facing staff in the research%sponsoring organisation will be able to help at the early stage of working outwhich attributes to measure. #hey understand the issues, they know the terminology and they will welcome beingconsulted. nternal focus groups with the sales staff will prove highly instructive. #his internally generatedinformation may be biased, but it will raise most of the general customer issues and is readily available at little cost.

    t is wise to cross check the internal views with a small number of depth interviews with customers. -alf a do7enmay be all that is re!uired.

    When all this work has been completed a list of attributes can be selected for rating.

    About price

    2arket price#otal cost of use8alue for money

    About the company

    3eputation of the company1ase of doing businessnvoice claritynvoices on time

    Aboutstaff and service

    +ourtesy from sales staff3epresentative9s availability3epresentative9s knowledge3eliability of returning calls$riendliness of the sales staff+omplaint resolution3esponsiveness to en!uiriesAfter sales service#echnical service

    About delivery

    0elivery on timepeed of delivery

    About the product

    :uality of the productength of life of the product0esign of the product+onsistency of !uality3ange of products;rocessibility of the product

  • 7/26/2019 Lllll Llllo Ooooooo

    3/176

    -W -/0 #-1

  • 7/26/2019 Lllll Llllo Ooooooo

    4/176

    When planning the fieldwork, there is likely to be a debate as to whether the interview should be carried out withoutdisclosing the identify of the sponsor. f the !uestions in the survey are about a particular company or product, it isobvious that the identity has to be disclosed. When the survey is carried out by phone or face to face, co%operationis helped if an advance letter is sent out e'plaining the purpose of the research. ogistically this may not be possiblein which case the e'planation for the survey would be built into the introductory script of the interviewer.

    f the survey covers a number of competing brands, disclosure of the research sponsor will bias the response. f theinterview is carried out anonymously, without disclosing the sponsor, bias will result through a considerably reducedstrike rate or guarded responses. #he interviewer, e'plaining at the outset of the interview that the sponsor will bedisclosed at the end of the interview, usually overcomes this.

    -W -/0 A#$A+#< "1 21A/310?

    +ustomers e'press their satisfaction in many ways. When they are satisfied, they mostly say nothing but returnagain and again to buy or use more. When asked how they feel about a company or its products in open%ended!uestioning they respond with anecdotes and may use terminology such as delighted, e'tremely satisfied, verydissatisfied etc. +ollecting the motleys variety of adectives together from open ended responses would beproblematical in a large survey. #o overcome this problem market researchers ask people to describe a companyusing verbal or numeric scales with words that measure attitudes.

    ;eople are used to the concept of rating things with numerical scores and these can work well in surveys. nce therespondent has been given the anchors of the scale, they can readily give a number to e'press their level ofsatisfaction. #ypically, scales of , @ or B are used where the lowest figure indicates e'treme dissatisfaction and thehighest shows e'treme satisfaction. #he stem of the scale is usually !uite short since a scale of up to BB wouldprove too demanding for rating the do7ens of specific issues that are often on the !uestionnaire.

    2easuring satisfaction is only half the story. t is also necessary to determine customers* e'pectations or theimportance they attach to the different attributes, otherwise resources could be spent raising satisfaction levels ofthings that do not matter. #he measurement of e'pectations or importance is more difficult than the measurement ofsatisfaction. 2any people do not know or cannot admit, even to themselves, what is important. +an believesomeone who says they bought a ;orsche for its 4engineering e'cellence5? +onsumers do not spend their timerationalising why they do things, their views change and they may not be able to easily communicate or admit to thecomple' issues in the buying argument.

    #he same interval scales of words or numbers are often used to measure importance , @ or B being veryimportant and being not at all important. -owever, most of the issues being researched are of some importance forotherwise they would not be considered in the study. As a result, the mean scores on importance may show littledifferentiation between the vital issues such as product !uality, price and delivery and the nice to have factors suchas knowledgeable representatives and long opening hours. 3anking can indicate the importance of a small list of upto si' or seven factors but respondents struggle to place things in rank order once the first four or five are out of theway. t would not work for determining the importance of CB attributes.

    As a check against factors that are given a 4stated importance5 score, researchers can statistically calculate (or4derive5) the importance of the same issues. 0erived importance is calculated by correlating the satisfaction levelsof each attribute with the overall level of satisfaction. Where there is a high link or correlation with an attribute, itcan be inferred that the attribute is driving customer satisfaction. 0eriving the importance of attributes can show the

    greater influence of softer issues such as the friendliness of the staff or the power of the brand things that peoplesomehow cannot rationalise or admit to in a 4stated5 answer.

  • 7/26/2019 Lllll Llllo Ooooooo

    5/176

    W-A# 0 #-1 21A/3121

  • 7/26/2019 Lllll Llllo Ooooooo

    6/176

    #his raises the interesting !uestion what is achievable and how far can we go in the pursuit of customersatisfaction. Abraham incoln*s !uote about fooling people could be usefully modified for customer satisfactionresearch to read 4=ou can satisfy all the people some of the time, and some of the people all the time, but you cannotsatisfy all the people all the time5. As marketers we know that we must segment our customer base. t is no goodtrying to satisfy everyone, as we do not aim our products at everyone. What matters is that we achieve high scoresof satisfaction in those segments in which we play. btaining scores of I or B from around a half to two thirds oftargeted customers on issues that are important to them should be the aim. ;lotting the customer satisfaction scoresagainst the importance score will show where the strengths and weaknesses lie, (see diagram E) with the mainobective to move all issues to the top right bo'.

    0iagram E6 (" gra)h to sho! !here customer satisfaction needs to im)rove

    -W # /1 +/#213 A#$A+#< #/01 # >31A#1# 1$$1+#

  • 7/26/2019 Lllll Llllo Ooooooo

    7/176

    methods introduced by >ordon "ethune, chairman and chief e'ecutive since IIG, is a NF cash bonus paid toemployees each month if +ontinental is among the most punctual carriers. t makes sense to reward, in ;avlovianfashion, immediately after the event and not si' or twelve months down the line when the effect will have beenforgotten.

    #wo companies, both ostensibly committed to customer satisfaction, but one markedly outperforming the other. #hecustomer satisfaction scores are only part of the story.

    A customer satisfaction inde' is a snapshot at a point in time. ;eople*s views change continuously and theperformance of companies in delivering customer satisfaction is also changing. 2easuring satisfaction must be acontinuous process. #racking surveys provide benchmarks of one*s own company*s performance and, if competitorsuppliers are also being measured, there will be measurements of relative performance. #his places considerableonus on the researcher to design a survey that will accurately show real differences, one survey to another. #he!uestionnaire needs to be consistent so there is no dispute about answers differing because of changes to !uestions.#he sample of each survey must be large enough to provide a reliable base and the selection of the sample mustmirror earlier surveys so like is being compared with like.

    "enchmarking in customer satisfaction can go beyond comparisons with direct competitors. ome firms have takenthis type of benchmarking a step further. nstead of ust developing a benchmark on competitors, they identify thebest firm in any industry at a particular activity. .. "ean may be benchmarked for telephone order processing or

    customer service. American 1'press may be benchmarked for billing and payment transactions.

    #here has been considerable research into the links between customer satisfaction and employee satisfaction Oaplan M

  • 7/26/2019 Lllll Llllo Ooooooo

    8/176

    0181; A< A+#< ;A< #-A# 31+#$1 W1AO

  • 7/26/2019 Lllll Llllo Ooooooo

    9/176

    !uality assurance, plus research and development. "ecause each department evaluates suppliers differently, thecustomer satisfaction programme will need to cover the multiple views.

    #he adage in market research that we turn to again and again is the need to ask the right !uestion of the right person.$inding that person in customer satisfaction research may re!uire a compromise with a focus on one person % the keydecision maker& perhaps the transport manager in the e'ample of the trucks. f money and time permit, differentpeople could be interviewed and this may involve different interviewing methods and different !uestions.

    #he traditional first in line customer is an obvious candidate for measuring customer satisfaction. "ut what aboutother people in the channel to market? f the products are sold through intermediaries, we are even further from ourcustomers. A good customer satisfaction program will include at least the most important of these types of channelcustomers, perhaps the wholesalers as well as the final consumers.

    ne of the greatest headaches in the organisation of a business to business customer satisfaction survey is thecompilation of the sample frame the list from which the sample of respondents is selected. "uilding an accurate,up%to%date list of customers, with telephone numbers and contact details is nearly always a challenge. #he list heldby the accounts department may not have the contact details of the people making the purchasing decision. argebusinesses may have regionally autonomous units and there may be some fiefdom that says it doesn*t want itscustomers pestered by market researchers. #he sales teams* +hristmas card lists may well be the best lists of all butthey are kept close to the chest of each sales person and not held on a central server. "uilding a good sample frame

    nearly always takes longer than was planned but it is the foundation of a good customer satisfaction survey.

    +ustomer satisfaction surveys are often ust that surveys of customers without consideration of the views of lost orpotential customers. apsed customers may have stories to tell about service issues while potential customers are agood source of benchmark data on the competition. f a survey is to embrace non%customers, the compilation of thesample frame is even more difficult. #he !uality of these sample frames influences the results more than any otherfactor since they are usually outside the researchers* control. #he !uestionnaire design and interpretation are withinthe control of the researchers and these are subects where they will have considerable e'perience.

  • 7/26/2019 Lllll Llllo Ooooooo

    10/176

    W-A# -/0 "1 21A/310?

    n customer satisfaction research we seek the views of respondents on a variety of issues that will show how thecompany is performing and how it can improve. #his understanding is obtained at a high level (4how satisfied areyou the A"+ td overall?5) and at a very specific level (4how satisfied are you with the clarity of invoices?5).

    -igh level issues are included in most customer satisfaction surveys and they could be captured by !uestions suchas6

    What is your overall satisfaction with A"+ td?-ow likely or unlikely are you to buy from A"+ td again?-ow likely or unlikely would you be to recommend A"+ td to a friend or colleague?

    t is at the more specific level of !uestioning that things become more difficult. ome issues are of obviousimportance and every supplier is e'pected to perform to a minimum acceptable level on them. #hese are thehygiene factors. f a company fails on any of these issues they would !uickly lose market share or go out ofbusiness. An airline must offer safety but the level of in%flight service is a variable. #hese variables such as in%flightservice are often the issues that differentiate companies and create the satisfaction or dissatisfaction.

    Working out what !uestions to ask at a detailed level means seeing the world from the customers* points of view.

    What do they consider important? #hese factors or attributes will differ from company to company and there couldbe a long list. #hey could include the following6

    About delivery

    0elivery on timepeed of delivery

    #he list is not e'haustive by any means. #here is no mention above of environmental issues, sales literature,fre!uency of representatives* calls or packaging. 1ven though the attributes are deemed specific, it is not entirelyclear what is meant by 4product !uality5 or 4ease of doing business5. +ryptic labels that summarise specific issueshave to be carefully chosen for otherwise it will be impossible to interpret the results.

    +ustomer facing staff in the research%sponsoring organisation will be able to help at the early stage of working outwhich attributes to measure. #hey understand the issues, they know the terminology and they will welcome beingconsulted. nternal focus groups with the sales staff will prove highly instructive. #his internally generatedinformation may be biased, but it will raise most of the general customer issues and is readily available at little cost.

    t is wise to cross check the internal views with a small number of depth interviews with customers. -alf a do7enmay be all that is re!uired.

    When all this work has been completed a list of attributes can be selected for rating.

    About price

    2arket price#otal cost of use8alue for money

    About the company

    3eputation of the company1ase of doing businessnvoice claritynvoices on time

    Aboutstaff and service

    +ourtesy from sales staff3epresentative9s availability3epresentative9s knowledge3eliability of returning calls$riendliness of the sales staff+omplaint resolution3esponsiveness to en!uiriesAfter sales service#echnical service

    About delivery

    0elivery on timepeed of delivery

    About the product

    :uality of the productength of life of the product0esign of the product+onsistency of !uality3ange of products;rocessibility of the product

  • 7/26/2019 Lllll Llllo Ooooooo

    11/176

    -W -/0 #-1

  • 7/26/2019 Lllll Llllo Ooooooo

    12/176

    When planning the fieldwork, there is likely to be a debate as to whether the interview should be carried out withoutdisclosing the identify of the sponsor. f the !uestions in the survey are about a particular company or product, it isobvious that the identity has to be disclosed. When the survey is carried out by phone or face to face, co%operationis helped if an advance letter is sent out e'plaining the purpose of the research. ogistically this may not be possiblein which case the e'planation for the survey would be built into the introductory script of the interviewer.

    f the survey covers a number of competing brands, disclosure of the research sponsor will bias the response. f theinterview is carried out anonymously, without disclosing the sponsor, bias will result through a considerably reducedstrike rate or guarded responses. #he interviewer, e'plaining at the outset of the interview that the sponsor will bedisclosed at the end of the interview, usually overcomes this.

    -W -/0 A#$A+#< "1 21A/310?

    +ustomers e'press their satisfaction in many ways. When they are satisfied, they mostly say nothing but returnagain and again to buy or use more. When asked how they feel about a company or its products in open%ended!uestioning they respond with anecdotes and may use terminology such as delighted, e'tremely satisfied, verydissatisfied etc. +ollecting the motleys variety of adectives together from open ended responses would beproblematical in a large survey. #o overcome this problem market researchers ask people to describe a companyusing verbal or numeric scales with words that measure attitudes.

    ;eople are used to the concept of rating things with numerical scores and these can work well in surveys. nce therespondent has been given the anchors of the scale, they can readily give a number to e'press their level ofsatisfaction. #ypically, scales of , @ or B are used where the lowest figure indicates e'treme dissatisfaction and thehighest shows e'treme satisfaction. #he stem of the scale is usually !uite short since a scale of up to BB wouldprove too demanding for rating the do7ens of specific issues that are often on the !uestionnaire.

    2easuring satisfaction is only half the story. t is also necessary to determine customers* e'pectations or theimportance they attach to the different attributes, otherwise resources could be spent raising satisfaction levels ofthings that do not matter. #he measurement of e'pectations or importance is more difficult than the measurement ofsatisfaction. 2any people do not know or cannot admit, even to themselves, what is important. +an believesomeone who says they bought a ;orsche for its 4engineering e'cellence5? +onsumers do not spend their timerationalising why they do things, their views change and they may not be able to easily communicate or admit to thecomple' issues in the buying argument.

    #he same interval scales of words or numbers are often used to measure importance , @ or B being veryimportant and being not at all important. -owever, most of the issues being researched are of some importance forotherwise they would not be considered in the study. As a result, the mean scores on importance may show littledifferentiation between the vital issues such as product !uality, price and delivery and the nice to have factors suchas knowledgeable representatives and long opening hours. 3anking can indicate the importance of a small list of upto si' or seven factors but respondents struggle to place things in rank order once the first four or five are out of theway. t would not work for determining the importance of CB attributes.

    As a check against factors that are given a 4stated importance5 score, researchers can statistically calculate (or4derive5) the importance of the same issues. 0erived importance is calculated by correlating the satisfaction levelsof each attribute with the overall level of satisfaction. Where there is a high link or correlation with an attribute, itcan be inferred that the attribute is driving customer satisfaction. 0eriving the importance of attributes can show the

    greater influence of softer issues such as the friendliness of the staff or the power of the brand things that peoplesomehow cannot rationalise or admit to in a 4stated5 answer.

  • 7/26/2019 Lllll Llllo Ooooooo

    13/176

    W-A# 0 #-1 21A/3121

  • 7/26/2019 Lllll Llllo Ooooooo

    14/176

    #his raises the interesting !uestion what is achievable and how far can we go in the pursuit of customersatisfaction. Abraham incoln*s !uote about fooling people could be usefully modified for customer satisfactionresearch to read 4=ou can satisfy all the people some of the time, and some of the people all the time, but you cannotsatisfy all the people all the time5. As marketers we know that we must segment our customer base. t is no goodtrying to satisfy everyone, as we do not aim our products at everyone. What matters is that we achieve high scoresof satisfaction in those segments in which we play. btaining scores of I or B from around a half to two thirds oftargeted customers on issues that are important to them should be the aim. ;lotting the customer satisfaction scoresagainst the importance score will show where the strengths and weaknesses lie, (see diagram E) with the mainobective to move all issues to the top right bo'.

    0iagram E6 (" gra)h to sho! !here customer satisfaction needs to im)rove

    -W # /1 +/#213 A#$A+#< #/01 # >31A#1# 1$$1+#

  • 7/26/2019 Lllll Llllo Ooooooo

    15/176

    methods introduced by >ordon "ethune, chairman and chief e'ecutive since IIG, is a NF cash bonus paid toemployees each month if +ontinental is among the most punctual carriers. t makes sense to reward, in ;avlovianfashion, immediately after the event and not si' or twelve months down the line when the effect will have beenforgotten.

    #wo companies, both ostensibly committed to customer satisfaction, but one markedly outperforming the other. #hecustomer satisfaction scores are only part of the story.

    A customer satisfaction inde' is a snapshot at a point in time. ;eople*s views change continuously and theperformance of companies in delivering customer satisfaction is also changing. 2easuring satisfaction must be acontinuous process. #racking surveys provide benchmarks of one*s own company*s performance and, if competitorsuppliers are also being measured, there will be measurements of relative performance. #his places considerableonus on the researcher to design a survey that will accurately show real differences, one survey to another. #he!uestionnaire needs to be consistent so there is no dispute about answers differing because of changes to !uestions.#he sample of each survey must be large enough to provide a reliable base and the selection of the sample mustmirror earlier surveys so like is being compared with like.

    "enchmarking in customer satisfaction can go beyond comparisons with direct competitors. ome firms have takenthis type of benchmarking a step further. nstead of ust developing a benchmark on competitors, they identify thebest firm in any industry at a particular activity. .. "ean may be benchmarked for telephone order processing or

    customer service. American 1'press may be benchmarked for billing and payment transactions.

    #here has been considerable research into the links between customer satisfaction and employee satisfaction Oaplan M

  • 7/26/2019 Lllll Llllo Ooooooo

    16/176

    0181; A< A+#< ;A< #-A# 31+#$1 W1AO

  • 7/26/2019 Lllll Llllo Ooooooo

    17/176

    !uality assurance, plus research and development. "ecause each department evaluates suppliers differently, thecustomer satisfaction programme will need to cover the multiple views.

    #he adage in market research that we turn to again and again is the need to ask the right !uestion of the right person.$inding that person in customer satisfaction research may re!uire a compromise with a focus on one person % the keydecision maker& perhaps the transport manager in the e'ample of the trucks. f money and time permit, differentpeople could be interviewed and this may involve different interviewing methods and different !uestions.

    #he traditional first in line customer is an obvious candidate for measuring customer satisfaction. "ut what aboutother people in the channel to market? f the products are sold through intermediaries, we are even further from ourcustomers. A good customer satisfaction program will include at least the most important of these types of channelcustomers, perhaps the wholesalers as well as the final consumers.

    ne of the greatest headaches in the organisation of a business to business customer satisfaction survey is thecompilation of the sample frame the list from which the sample of respondents is selected. "uilding an accurate,up%to%date list of customers, with telephone numbers and contact details is nearly always a challenge. #he list heldby the accounts department may not have the contact details of the people making the purchasing decision. argebusinesses may have regionally autonomous units and there may be some fiefdom that says it doesn*t want itscustomers pestered by market researchers. #he sales teams* +hristmas card lists may well be the best lists of all butthey are kept close to the chest of each sales person and not held on a central server. "uilding a good sample frame

    nearly always takes longer than was planned but it is the foundation of a good customer satisfaction survey.

    +ustomer satisfaction surveys are often ust that surveys of customers without consideration of the views of lost orpotential customers. apsed customers may have stories to tell about service issues while potential customers are agood source of benchmark data on the competition. f a survey is to embrace non%customers, the compilation of thesample frame is even more difficult. #he !uality of these sample frames influences the results more than any otherfactor since they are usually outside the researchers* control. #he !uestionnaire design and interpretation are withinthe control of the researchers and these are subects where they will have considerable e'perience.

  • 7/26/2019 Lllll Llllo Ooooooo

    18/176

    W-A# -/0 "1 21A/310?

    n customer satisfaction research we seek the views of respondents on a variety of issues that will show how thecompany is performing and how it can improve. #his understanding is obtained at a high level (4how satisfied areyou the A"+ td overall?5) and at a very specific level (4how satisfied are you with the clarity of invoices?5).

    -igh level issues are included in most customer satisfaction surveys and they could be captured by !uestions suchas6

    What is your overall satisfaction with A"+ td?-ow likely or unlikely are you to buy from A"+ td again?-ow likely or unlikely would you be to recommend A"+ td to a friend or colleague?

    t is at the more specific level of !uestioning that things become more difficult. ome issues are of obviousimportance and every supplier is e'pected to perform to a minimum acceptable level on them. #hese are thehygiene factors. f a company fails on any of these issues they would !uickly lose market share or go out ofbusiness. An airline must offer safety but the level of in%flight service is a variable. #hese variables such as in%flightservice are often the issues that differentiate companies and create the satisfaction or dissatisfaction.

    Working out what !uestions to ask at a detailed level means seeing the world from the customers* points of view.

    What do they consider important? #hese factors or attributes will differ from company to company and there couldbe a long list. #hey could include the following6

    About delivery

    0elivery on timepeed of delivery

    #he list is not e'haustive by any means. #here is no mention above of environmental issues, sales literature,fre!uency of representatives* calls or packaging. 1ven though the attributes are deemed specific, it is not entirelyclear what is meant by 4product !uality5 or 4ease of doing business5. +ryptic labels that summarise specific issueshave to be carefully chosen for otherwise it will be impossible to interpret the results.

    +ustomer facing staff in the research%sponsoring organisation will be able to help at the early stage of working outwhich attributes to measure. #hey understand the issues, they know the terminology and they will welcome beingconsulted. nternal focus groups with the sales staff will prove highly instructive. #his internally generatedinformation may be biased, but it will raise most of the general customer issues and is readily available at little cost.

    t is wise to cross check the internal views with a small number of depth interviews with customers. -alf a do7enmay be all that is re!uired.

    When all this work has been completed a list of attributes can be selected for rating.

    About price

    2arket price#otal cost of use8alue for money

    About the company

    3eputation of the company1ase of doing businessnvoice claritynvoices on time

    Aboutstaff and service

    +ourtesy from sales staff3epresentative9s availability3epresentative9s knowledge3eliability of returning calls$riendliness of the sales staff+omplaint resolution3esponsiveness to en!uiriesAfter sales service#echnical service

    About delivery

    0elivery on timepeed of delivery

    About the product

    :uality of the productength of life of the product0esign of the product+onsistency of !uality3ange of products;rocessibility of the product

  • 7/26/2019 Lllll Llllo Ooooooo

    19/176

    -W -/0 #-1

  • 7/26/2019 Lllll Llllo Ooooooo

    20/176

    When planning the fieldwork, there is likely to be a debate as to whether the interview should be carried out withoutdisclosing the identify of the sponsor. f the !uestions in the survey are about a particular company or product, it isobvious that the identity has to be disclosed. When the survey is carried out by phone or face to face, co%operationis helped if an advance letter is sent out e'plaining the purpose of the research. ogistically this may not be possiblein which case the e'planation for the survey would be built into the introductory script of the interviewer.

    f the survey covers a number of competing brands, disclosure of the research sponsor will bias the response. f theinterview is carried out anonymously, without disclosing the sponsor, bias will result through a considerably reducedstrike rate or guarded responses. #he interviewer, e'plaining at the outset of the interview that the sponsor will bedisclosed at the end of the interview, usually overcomes this.

    -W -/0 A#$A+#< "1 21A/310?

    +ustomers e'press their satisfaction in many ways. When they are satisfied, they mostly say nothing but returnagain and again to buy or use more. When asked how they feel about a company or its products in open%ended!uestioning they respond with anecdotes and may use terminology such as delighted, e'tremely satisfied, verydissatisfied etc. +ollecting the motleys variety of adectives together from open ended responses would beproblematical in a large survey. #o overcome this problem market researchers ask people to describe a companyusing verbal or numeric scales with words that measure attitudes.

    ;eople are used to the concept of rating things with numerical scores and these can work well in surveys. nce therespondent has been given the anchors of the scale, they can readily give a number to e'press their level ofsatisfaction. #ypically, scales of , @ or B are used where the lowest figure indicates e'treme dissatisfaction and thehighest shows e'treme satisfaction. #he stem of the scale is usually !uite short since a scale of up to BB wouldprove too demanding for rating the do7ens of specific issues that are often on the !uestionnaire.

    2easuring satisfaction is only half the story. t is also necessary to determine customers* e'pectations or theimportance they attach to the different attributes, otherwise resources could be spent raising satisfaction levels ofthings that do not matter. #he measurement of e'pectations or importance is more difficult than the measurement ofsatisfaction. 2any people do not know or cannot admit, even to themselves, what is important. +an believesomeone who says they bought a ;orsche for its 4engineering e'cellence5? +onsumers do not spend their timerationalising why they do things, their views change and they may not be able to easily communicate or admit to thecomple' issues in the buying argument.

    #he same interval scales of words or numbers are often used to measure importance , @ or B being veryimportant and being not at all important. -owever, most of the issues being researched are of some importance forotherwise they would not be considered in the study. As a result, the mean scores on importance may show littledifferentiation between the vital issues such as product !uality, price and delivery and the nice to have factors suchas knowledgeable representatives and long opening hours. 3anking can indicate the importance of a small list of upto si' or seven factors but respondents struggle to place things in rank order once the first four or five are out of theway. t would not work for determining the importance of CB attributes.

    As a check against factors that are given a 4stated importance5 score, researchers can statistically calculate (or4derive5) the importance of the same issues. 0erived importance is calculated by correlating the satisfaction levelsof each attribute with the overall level of satisfaction. Where there is a high link or correlation with an attribute, itcan be inferred that the attribute is driving customer satisfaction. 0eriving the importance of attributes can show the

    greater influence of softer issues such as the friendliness of the staff or the power of the brand things that peoplesomehow cannot rationalise or admit to in a 4stated5 answer.

  • 7/26/2019 Lllll Llllo Ooooooo

    21/176

    W-A# 0 #-1 21A/3121

  • 7/26/2019 Lllll Llllo Ooooooo

    22/176

    #his raises the interesting !uestion what is achievable and how far can we go in the pursuit of customersatisfaction. Abraham incoln*s !uote about fooling people could be usefully modified for customer satisfactionresearch to read 4=ou can satisfy all the people some of the time, and some of the people all the time, but you cannotsatisfy all the people all the time5. As marketers we know that we must segment our customer base. t is no goodtrying to satisfy everyone, as we do not aim our products at everyone. What matters is that we achieve high scoresof satisfaction in those segments in which we play. btaining scores of I or B from around a half to two thirds oftargeted customers on issues that are important to them should be the aim. ;lotting the customer satisfaction scoresagainst the importance score will show where the strengths and weaknesses lie, (see diagram E) with the mainobective to move all issues to the top right bo'.

    0iagram E6 (" gra)h to sho! !here customer satisfaction needs to im)rove

    -W # /1 +/#213 A#$A+#< #/01 # >31A#1# 1$$1+#

  • 7/26/2019 Lllll Llllo Ooooooo

    23/176

    methods introduced by >ordon "ethune, chairman and chief e'ecutive since IIG, is a NF cash bonus paid toemployees each month if +ontinental is among the most punctual carriers. t makes sense to reward, in ;avlovianfashion, immediately after the event and not si' or twelve months down the line when the effect will have beenforgotten.

    #wo companies, both ostensibly committed to customer satisfaction, but one markedly outperforming the other. #hecustomer satisfaction scores are only part of the story.

    A customer satisfaction inde' is a snapshot at a point in time. ;eople*s views change continuously and theperformance of companies in delivering customer satisfaction is also changing. 2easuring satisfaction must be acontinuous process. #racking surveys provide benchmarks of one*s own company*s performance and, if competitorsuppliers are also being measured, there will be measurements of relative performance. #his places considerableonus on the researcher to design a survey that will accurately show real differences, one survey to another. #he!uestionnaire needs to be consistent so there is no dispute about answers differing because of changes to !uestions.#he sample of each survey must be large enough to provide a reliable base and the selection of the sample mustmirror earlier surveys so like is being compared with like.

    "enchmarking in customer satisfaction can go beyond comparisons with direct competitors. ome firms have takenthis type of benchmarking a step further. nstead of ust developing a benchmark on competitors, they identify thebest firm in any industry at a particular activity. .. "ean may be benchmarked for telephone order processing or

    customer service. American 1'press may be benchmarked for billing and payment transactions.

    #here has been considerable research into the links between customer satisfaction and employee satisfaction Oaplan M

  • 7/26/2019 Lllll Llllo Ooooooo

    24/176

    0181; A< A+#< ;A< #-A# 31+#$1 W1AO

  • 7/26/2019 Lllll Llllo Ooooooo

    25/176

    !uality assurance, plus research and development. "ecause each department evaluates suppliers differently, thecustomer satisfaction programme will need to cover the multiple views.

    #he adage in market research that we turn to again and again is the need to ask the right !uestion of the right person.$inding that person in customer satisfaction research may re!uire a compromise with a focus on one person % the keydecision maker& perhaps the transport manager in the e'ample of the trucks. f money and time permit, differentpeople could be interviewed and this may involve different interviewing methods and different !uestions.

    #he traditional first in line customer is an obvious candidate for measuring customer satisfaction. "ut what aboutother people in the channel to market? f the products are sold through intermediaries, we are even further from ourcustomers. A good customer satisfaction program will include at least the most important of these types of channelcustomers, perhaps the wholesalers as well as the final consumers.

    ne of the greatest headaches in the organisation of a business to business customer satisfaction survey is thecompilation of the sample frame the list from which the sample of respondents is selected. "uilding an accurate,up%to%date list of customers, with telephone numbers and contact details is nearly always a challenge. #he list heldby the accounts department may not have the contact details of the people making the purchasing decision. argebusinesses may have regionally autonomous units and there may be some fiefdom that says it doesn*t want itscustomers pestered by market researchers. #he sales teams* +hristmas card lists may well be the best lists of all butthey are kept close to the chest of each sales person and not held on a central server. "uilding a good sample frame

    nearly always takes longer than was planned but it is the foundation of a good customer satisfaction survey.

    +ustomer satisfaction surveys are often ust that surveys of customers without consideration of the views of lost orpotential customers. apsed customers may have stories to tell about service issues while potential customers are agood source of benchmark data on the competition. f a survey is to embrace non%customers, the compilation of thesample frame is even more difficult. #he !uality of these sample frames influences the results more than any otherfactor since they are usually outside the researchers* control. #he !uestionnaire design and interpretation are withinthe control of the researchers and these are subects where they will have considerable e'perience.

  • 7/26/2019 Lllll Llllo Ooooooo

    26/176

    W-A# -/0 "1 21A/310?

    n customer satisfaction research we seek the views of respondents on a variety of issues that will show how thecompany is performing and how it can improve. #his understanding is obtained at a high level (4how satisfied areyou the A"+ td overall?5) and at a very specific level (4how satisfied are you with the clarity of invoices?5).

    -igh level issues are included in most customer satisfaction surveys and they could be captured by !uestions suchas6

    What is your overall satisfaction with A"+ td?-ow likely or unlikely are you to buy from A"+ td again?-ow likely or unlikely would you be to recommend A"+ td to a friend or colleague?

    t is at the more specific level of !uestioning that things become more difficult. ome issues are of obviousimportance and every supplier is e'pected to perform to a minimum acceptable level on them. #hese are thehygiene factors. f a company fails on any of these issues they would !uickly lose market share or go out ofbusiness. An airline must offer safety but the level of in%flight service is a variable. #hese variables such as in%flightservice are often the issues that differentiate companies and create the satisfaction or dissatisfaction.

    Working out what !uestions to ask at a detailed level means seeing the world from the customers* points of view.

    What do they consider important? #hese factors or attributes will differ from company to company and there couldbe a long list. #hey could include the following6

    About delivery

    0elivery on timepeed of delivery

    #he list is not e'haustive by any means. #here is no mention above of environmental issues, sales literature,fre!uency of representatives* calls or packaging. 1ven though the attributes are deemed specific, it is not entirelyclear what is meant by 4product !uality5 or 4ease of doing business5. +ryptic labels that summarise specific issueshave to be carefully chosen for otherwise it will be impossible to interpret the results.

    +ustomer facing staff in the research%sponsoring organisation will be able to help at the early stage of working outwhich attributes to measure. #hey understand the issues, they know the terminology and they will welcome beingconsulted. nternal focus groups with the sales staff will prove highly instructive. #his internally generatedinformation may be biased, but it will raise most of the general customer issues and is readily available at little cost.

    t is wise to cross check the internal views with a small number of depth interviews with customers. -alf a do7enmay be all that is re!uired.

    When all this work has been completed a list of attributes can be selected for rating.

    About price

    2arket price#otal cost of use8alue for money

    About the company

    3eputation of the company1ase of doing businessnvoice claritynvoices on time

    Aboutstaff and service

    +ourtesy from sales staff3epresentative9s availability3epresentative9s knowledge3eliability of returning calls$riendliness of the sales staff+omplaint resolution3esponsiveness to en!uiriesAfter sales service#echnical service

    About delivery

    0elivery on timepeed of delivery

    About the product

    :uality of the productength of life of the product0esign of the product+onsistency of !uality3ange of products;rocessibility of the product

  • 7/26/2019 Lllll Llllo Ooooooo

    27/176

    -W -/0 #-1

  • 7/26/2019 Lllll Llllo Ooooooo

    28/176

    When planning the fieldwork, there is likely to be a debate as to whether the interview should be carried out withoutdisclosing the identify of the sponsor. f the !uestions in the survey are about a particular company or product, it isobvious that the identity has to be disclosed. When the survey is carried out by phone or face to face, co%operationis helped if an advance letter is sent out e'plaining the purpose of the research. ogistically this may not be possiblein which case the e'planation for the survey would be built into the introductory script of the interviewer.

    f the survey covers a number of competing brands, disclosure of the research sponsor will bias the response. f theinterview is carried out anonymously, without disclosing the sponsor, bias will result through a considerably reducedstrike rate or guarded responses. #he interviewer, e'plaining at the outset of the interview that the sponsor will bedisclosed at the end of the interview, usually overcomes this.

    -W -/0 A#$A+#< "1 21A/310?

    +ustomers e'press their satisfaction in many ways. When they are satisfied, they mostly say nothing but returnagain and again to buy or use more. When asked how they feel about a company or its products in open%ended!uestioning they respond with anecdotes and may use terminology such as delighted, e'tremely satisfied, verydissatisfied etc. +ollecting the motleys variety of adectives together from open ended responses would beproblematical in a large survey. #o overcome this problem market researchers ask people to describe a companyusing verbal or numeric scales with words that measure attitudes.

    ;eople are used to the concept of rating things with numerical scores and these can work well in surveys. nce therespondent has been given the anchors of the scale, they can readily give a number to e'press their level ofsatisfaction. #ypically, scales of , @ or B are used where the lowest figure indicates e'treme dissatisfaction and thehighest shows e'treme satisfaction. #he stem of the scale is usually !uite short since a scale of up to BB wouldprove too demanding for rating the do7ens of specific issues that are often on the !uestionnaire.

    2easuring satisfaction is only half the story. t is also necessary to determine customers* e'pectations or theimportance they attach to the different attributes, otherwise resources could be spent raising satisfaction levels ofthings that do not matter. #he measurement of e'pectations or importance is more difficult than the measurement ofsatisfaction. 2any people do not know or cannot admit, even to themselves, what is important. +an believesomeone who says they bought a ;orsche for its 4engineering e'cellence5? +onsumers do not spend their timerationalising why they do things, their views change and they may not be able to easily communicate or admit to thecomple' issues in the buying argument.

    #he same interval scales of words or numbers are often used to measure importance , @ or B being veryimportant and being not at all important. -owever, most of the issues being researched are of some importance forotherwise they would not be considered in the study. As a result, the mean scores on importance may show littledifferentiation between the vital issues such as product !uality, price and delivery and the nice to have factors suchas knowledgeable representatives and long opening hours. 3anking can indicate the importance of a small list of upto si' or seven factors but respondents struggle to place things in rank order once the first four or five are out of theway. t would not work for determining the importance of CB attributes.

    As a check against factors that are given a 4stated importance5 score, researchers can statistically calculate (or4derive5) the importance of the same issues. 0erived importance is calculated by correlating the satisfaction levelsof each attribute with the overall level of satisfaction. Where there is a high link or correlation with an attribute, itcan be inferred that the attribute is driving customer satisfaction. 0eriving the importance of attributes can show the

    greater influence of softer issues such as the friendliness of the staff or the power of the brand things that peoplesomehow cannot rationalise or admit to in a 4stated5 answer.

  • 7/26/2019 Lllll Llllo Ooooooo

    29/176

    W-A# 0 #-1 21A/3121

  • 7/26/2019 Lllll Llllo Ooooooo

    30/176

    #his raises the interesting !uestion what is achievable and how far can we go in the pursuit of customersatisfaction. Abraham incoln*s !uote about fooling people could be usefully modified for customer satisfactionresearch to read 4=ou can satisfy all the people some of the time, and some of the people all the time, but you cannotsatisfy all the people all the time5. As marketers we know that we must segment our customer base. t is no goodtrying to satisfy everyone, as we do not aim our products at everyone. What matters is that we achieve high scoresof satisfaction in those segments in which we play. btaining scores of I or B from around a half to two thirds oftargeted customers on issues that are important to them should be the aim. ;lotting the customer satisfaction scoresagainst the importance score will show where the strengths and weaknesses lie, (see diagram E) with the mainobective to move all issues to the top right bo'.

    0iagram E6 (" gra)h to sho! !here customer satisfaction needs to im)rove

    -W # /1 +/#213 A#$A+#< #/01 # >31A#1# 1$$1+#

  • 7/26/2019 Lllll Llllo Ooooooo

    31/176

    methods introduced by >ordon "ethune, chairman and chief e'ecutive since IIG, is a NF cash bonus paid toemployees each month if +ontinental is among the most punctual carriers. t makes sense to reward, in ;avlovianfashion, immediately after the event and not si' or twelve months down the line when the effect will have beenforgotten.

    #wo companies, both ostensibly committed to customer satisfaction, but one markedly outperforming the other. #hecustomer satisfaction scores are only part of the story.

    A customer satisfaction inde' is a snapshot at a point in time. ;eople*s views change continuously and theperformance of companies in delivering customer satisfaction is also changing. 2easuring satisfaction must be acontinuous process. #racking surveys provide benchmarks of one*s own company*s performance and, if competitorsuppliers are also being measured, there will be measurements of relative performance. #his places considerableonus on the researcher to design a survey that will accurately show real differences, one survey to another. #he!uestionnaire needs to be consistent so there is no dispute about answers differing because of changes to !uestions.#he sample of each survey must be large enough to provide a reliable base and the selection of the sample mustmirror earlier surveys so like is being compared with like.

    "enchmarking in customer satisfaction can go beyond comparisons with direct competitors. ome firms have takenthis type of benchmarking a step further. nstead of ust developing a benchmark on competitors, they identify thebest firm in any industry at a particular activity. .. "ean may be benchmarked for telephone order processing or

    customer service. American 1'press may be benchmarked for billing and payment transactions.

    #here has been considerable research into the links between customer satisfaction and employee satisfaction Oaplan M

  • 7/26/2019 Lllll Llllo Ooooooo

    32/176

    0181; A< A+#< ;A< #-A# 31+#$1 W1AO

  • 7/26/2019 Lllll Llllo Ooooooo

    33/176

    !uality assurance, plus research and development. "ecause each department evaluates suppliers differently, thecustomer satisfaction programme will need to cover the multiple views.

    #he adage in market research that we turn to again and again is the need to ask the right !uestion of the right person.$inding that person in customer satisfaction research may re!uire a compromise with a focus on one person % the keydecision maker& perhaps the transport manager in the e'ample of the trucks. f money and time permit, differentpeople could be interviewed and this may involve different interviewing methods and different !uestions.

    #he traditional first in line customer is an obvious candidate for measuring customer satisfaction. "ut what aboutother people in the channel to market? f the products are sold through intermediaries, we are even further from ourcustomers. A good customer satisfaction program will include at least the most important of these types of channelcustomers, perhaps the wholesalers as well as the final consumers.

    ne of the greatest headaches in the organisation of a business to business customer satisfaction survey is thecompilation of the sample frame the list from which the sample of respondents is selected. "uilding an accurate,up%to%date list of customers, with telephone numbers and contact details is nearly always a challenge. #he list heldby the accounts department may not have the contact details of the people making the purchasing decision. argebusinesses may have regionally autonomous units and there may be some fiefdom that says it doesn*t want itscustomers pestered by market researchers. #he sales teams* +hristmas card lists may well be the best lists of all butthey are kept close to the chest of each sales person and not held on a central server. "uilding a good sample frame

    nearly always takes longer than was planned but it is the foundation of a good customer satisfaction survey.

    +ustomer satisfaction surveys are often ust that surveys of customers without consideration of the views of lost orpotential customers. apsed customers may have stories to tell about service issues while potential customers are agood source of benchmark data on the competition. f a survey is to embrace non%customers, the compilation of thesample frame is even more difficult. #he !uality of these sample frames influences the results more than any otherfactor since they are usually outside the researchers* control. #he !uestionnaire design and interpretation are withinthe control of the researchers and these are subects where they will have considerable e'perience.

  • 7/26/2019 Lllll Llllo Ooooooo

    34/176

    W-A# -/0 "1 21A/310?

    n customer satisfaction research we seek the views of respondents on a variety of issues that will show how thecompany is performing and how it can improve. #his understanding is obtained at a high level (4how satisfied areyou the A"+ td overall?5) and at a very specific level (4how satisfied are you with the clarity of invoices?5).

    -igh level issues are included in most customer satisfaction surveys and they could be captured by !uestions suchas6

    What is your overall satisfaction with A"+ td?-ow likely or unlikely are you to buy from A"+ td again?-ow likely or unlikely would you be to recommend A"+ td to a friend or colleague?

    t is at the more specific level of !uestioning that things become more difficult. ome issues are of obviousimportance and every supplier is e'pected to perform to a minimum acceptable level on them. #hese are thehygiene factors. f a company fails on any of these issues they would !uickly lose market share or go out ofbusiness. An airline must offer safety but the level of in%flight service is a variable. #hese variables such as in%flightservice are often the issues that differentiate companies and create the satisfaction or dissatisfaction.

    Working out what !uestions to ask at a detailed level means seeing the world from the customers* points of view.

    What do they consider important? #hese factors or attributes will differ from company to company and there couldbe a long list. #hey could include the following6

    About delivery

    0elivery on timepeed of delivery

    #he list is not e'haustive by any means. #here is no mention above of environmental issues, sales literature,fre!uency of representatives* calls or packaging. 1ven though the attributes are deemed specific, it is not entirelyclear what is meant by 4product !uality5 or 4ease of doing business5. +ryptic labels that summarise specific issueshave to be carefully chosen for otherwise it will be impossible to interpret the results.

    +ustomer facing staff in the research%sponsoring organisation will be able to help at the early stage of working outwhich attributes to measure. #hey understand the issues, they know the terminology and they will welcome beingconsulted. nternal focus groups with the sales staff will prove highly instructive. #his internally generatedinformation may be biased, but it will raise most of the general customer issues and is readily available at little cost.

    t is wise to cross check the internal views with a small number of depth interviews with customers. -alf a do7enmay be all that is re!uired.

    When all this work has been completed a list of attributes can be selected for rating.

    About price

    2arket price#otal cost of use8alue for money

    About the company

    3eputation of the company1ase of doing businessnvoice claritynvoices on time

    Aboutstaff and service

    +ourtesy from sales staff3epresentative9s availability3epresentative9s knowledge3eliability of returning calls$riendliness of the sales staff+omplaint resolution3esponsiveness to en!uiriesAfter sales service#echnical service

    About delivery

    0elivery on timepeed of delivery

    About the product

    :uality of the productength of life of the product0esign of the product+onsistency of !uality3ange of products;rocessibility of the product

  • 7/26/2019 Lllll Llllo Ooooooo

    35/176

    -W -/0 #-1

  • 7/26/2019 Lllll Llllo Ooooooo

    36/176

    When planning the fieldwork, there is likely to be a debate as to whether the interview should be carried out withoutdisclosing the identify of the sponsor. f the !uestions in the survey are about a particular company or product, it isobvious that the identity has to be disclosed. When the survey is carried out by phone or face to face, co%operationis helped if an advance letter is sent out e'plaining the purpose of the research. ogistically this may not be possiblein which case the e'planation for the survey would be built into the introductory script of the interviewer.

    f the survey covers a number of competing brands, disclosure of the research sponsor will bias the response. f theinterview is carried out anonymously, without disclosing the sponsor, bias will result through a considerably reducedstrike rate or guarded responses. #he interviewer, e'plaining at the outset of the interview that the sponsor will bedisclosed at the end of the interview, usually overcomes this.

    -W -/0 A#$A+#< "1 21A/310?

    +ustomers e'press their satisfaction in many ways. When they are satisfied, they mostly say nothing but returnagain and again to buy or use more. When asked how they feel about a company or its products in open%ended!uestioning they respond with anecdotes and may use terminology such as delighted, e'tremely satisfied, verydissatisfied etc. +ollecting the motleys variety of adectives together from open ended responses would beproblematical in a large survey. #o overcome this problem market researchers ask people to describe a companyusing verbal or numeric scales with words that measure attitudes.

    ;eople are used to the concept of rating things with numerical scores and these can work well in surveys. nce therespondent has been given the anchors of the scale, they can readily give a number to e'press their level ofsatisfaction. #ypically, scales of , @ or B are used where the lowest figure indicates e'treme dissatisfaction and thehighest shows e'treme satisfaction. #he stem of the scale is usually !uite short since a scale of up to BB wouldprove too demanding for rating the do7ens of specific issues that are often on the !uestionnaire.

    2easuring satisfaction is only half the story. t is also necessary to determine customers* e'pectations or theimportance they attach to the different attributes, otherwise resources could be spent raising satisfaction levels ofthings that do not matter. #he measurement of e'pectations or importance is more difficult than the measurement ofsatisfaction. 2any people do not know or cannot admit, even to themselves, what is important. +an believesomeone who says they bought a ;orsche for its 4engineering e'cellence5? +onsumers do not spend their timerationalising why they do things, their views change and they may not be able to easily communicate or admit to thecomple' issues in the buying argument.

    #he same interval scales of words or numbers are often used to measure importance , @ or B being veryimportant and being not at all important. -owever, most of the issues being researched are of some importance forotherwise they would not be considered in the study. As a result, the mean scores on importance may show littledifferentiation between the vital issues such as product !uality, price and delivery and the nice to have factors suchas knowledgeable representatives and long opening hours. 3anking can indicate the importance of a small list of upto si' or seven factors but respondents struggle to place things in rank order once the first four or five are out of theway. t would not work for determining the importance of CB attributes.

    As a check against factors that are given a 4stated importance5 score, researchers can statistically calculate (or4derive5) the importance of the same issues. 0erived importance is calculated by correlating the satisfaction levelsof each attribute with the overall level of satisfaction. Where there is a high link or correlation with an attribute, itcan be inferred that the attribute is driving customer satisfaction. 0eriving the importance of attributes can show the

    greater influence of softer issues such as the friendliness of the staff or the power of the brand things that peoplesomehow cannot rationalise or admit to in a 4stated5 answer.

  • 7/26/2019 Lllll Llllo Ooooooo

    37/176

    W-A# 0 #-1 21A/3121

  • 7/26/2019 Lllll Llllo Ooooooo

    38/176

    #his raises the interesting !uestion what is achievable and how far can we go in the pursuit of customersatisfaction. Abraham incoln*s !uote about fooling people could be usefully modified for customer satisfactionresearch to read 4=ou can satisfy all the people some of the time, and some of the people all the time, but you cannotsatisfy all the people all the time5. As marketers we know that we must segment our customer base. t is no goodtrying to satisfy everyone, as we do not aim our products at everyone. What matters is that we achieve high scoresof satisfaction in those segments in which we play. btaining scores of I or B from around a half to two thirds oftargeted customers on issues that are important to them should be the aim. ;lotting the customer satisfaction scoresagainst the importance score will show where the strengths and weaknesses lie, (see diagram E) with the mainobective to move all issues to the top right bo'.

    0iagram E6 (" gra)h to sho! !here customer satisfaction needs to im)rove

    -W # /1 +/#213 A#$A+#< #/01 # >31A#1# 1$$1+#

  • 7/26/2019 Lllll Llllo Ooooooo

    39/176

    methods introduced by >ordon "ethune, chairman and chief e'ecutive since IIG, is a NF cash bonus paid toemployees each month if +ontinental is among the most punctual carriers. t makes sense to reward, in ;avlovianfashion, immediately after the event and not si' or twelve months down the line when the effect will have beenforgotten.

    #wo companies, both ostensibly committed to customer satisfaction, but one markedly outperforming the other. #hecustomer satisfaction scores are only part of the story.

    A customer satisfaction inde' is a snapshot at a point in time. ;eople*s views change continuously and theperformance of companies in delivering customer satisfaction is also changing. 2easuring satisfaction must be acontinuous process. #racking surveys provide benchmarks of one*s own company*s performance and, if competitorsuppliers are also being measured, there will be measurements of relative performance. #his places considerableonus on the researcher to design a survey that will accurately show real differences, one survey to another. #he!uestionnaire needs to be consistent so there is no dispute about answers differing because of changes to !uestions.#he sample of each survey must be large enough to provide a reliable base and the selection of the sample mustmirror earlier surveys so like is being compared with like.

    "enchmarking in customer satisfaction can go beyond comparisons with direct competitors. ome firms have takenthis type of benchmarking a step further. nstead of ust developing a benchmark on competitors, they identify thebest firm in any industry at a particular activity. .. "ean may be benchmarked for telephone order processing or

    customer service. American 1'press may be benchmarked for billing and payment transactions.

    #here has been considerable research into the links between customer satisfaction and employee satisfaction Oaplan M

  • 7/26/2019 Lllll Llllo Ooooooo

    40/176

    0181; A< A+#< ;A< #-A# 31+#$1 W1AO

  • 7/26/2019 Lllll Llllo Ooooooo

    41/176

    !uality assurance, plus research and development. "ecause each department evaluates suppliers differently, thecustomer satisfaction programme will need to cover the multiple views.

    #he adage in market research that we turn to again and again is the need to ask the right !uestion of the right person.$inding that person in customer satisfaction research may re!uire a compromise with a focus on one person % the keydecision maker& perhaps the transport manager in the e'ample of the trucks. f money and time permit, differentpeople could be interviewed and this may involve different interviewing methods and different !uestions.

    #he traditional first in line customer is an obvious candidate for measuring customer satisfaction. "ut what aboutother people in the channel to market? f the products are sold through intermediaries, we are even further from ourcustomers. A good customer satisfaction program will include at least the most important of these types of channelcustomers, perhaps the wholesalers as well as the final consumers.

    ne of the greatest headaches in the organisation of a business to business customer satisfaction survey is thecompilation of the sample frame the list from which the sample of respondents is selected. "uilding an accurate,up%to%date list of customers, with telephone numbers and contact details is nearly always a challenge. #he list heldby the accounts department may not have the contact details of the people making the purchasing decision. argebusinesses may have regionally autonomous units and there may be some fiefdom that says it doesn*t want itscustomers pestered by market researchers. #he sales teams* +hristmas card lists may well be the best lists of all butthey are kept close to the chest of each sales person and not held on a central server. "uilding a good sample frame

    nearly always takes longer than was planned but it is the foundation of a good customer satisfaction survey.

    +ustomer satisfaction surveys are often ust that surveys of customers without consideration of the views of lost orpotential customers. apsed customers may have stories to tell about service issues while potential customers are agood source of benchmark data on the competition. f a survey is to embrace non%customers, the compilation of thesample frame is even more difficult. #he !uality of these sample frames influences the results more than any otherfactor since they are usually outside the researchers* control. #he !uestionnaire design and interpretation are withinthe control of the researchers and these are subects where they will have considerable e'perience.

  • 7/26/2019 Lllll Llllo Ooooooo

    42/176

    W-A# -/0 "1 21A/310?

    n customer satisfaction research we seek the views of respondents on a variety of issues that will show how thecompany is performing and how it can improve. #his understanding is obtained at a high level (4how satisfied areyou the A"+ td overall?5) and at a very specific level (4how satisfied are you with the clarity of invoices?5).

    -igh level issues are included in most customer satisfaction surveys and they could be captured by !uestions suchas6

    What is your overall satisfaction with A"+ td?-ow likely or unlikely are you to buy from A"+ td again?-ow likely or unlikely would you be to recommend A"+ td to a friend or colleague?

    t is at the more specific level of !uestioning that things become more difficult. ome issues are of obviousimportance and every supplier is e'pected to perform to a minimum acceptable level on them. #hese are thehygiene factors. f a company fails on any of these issues they would !uickly lose market share or go out ofbusiness. An airline must offer safety but the level of in%flight service is a variable. #hese variables such as in%flightservice are often the issues that differentiate companies and create the satisfaction or dissatisfaction.

    Working out what !uestions to ask at a detailed level means seeing the world from the customers* points of view.

    What do they consider important? #hese factors or attributes will differ from company to company and there couldbe a long list. #hey could include the following6

    About delivery

    0elivery on timepeed of delivery

    #he list is not e'haustive by any means. #here is no mention above of environmental issues, sales literature,fre!uency of representatives* calls or packaging. 1ven though the attributes are deemed specific, it is not entirelyclear what is meant by 4product !uality5 or 4ease of doing business5. +ryptic labels that summarise specific issueshave to be carefully chosen for otherwise it will be impossible to interpret the results.

    +ustomer facing staff in the research%sponsoring organisation will be able to help at the early stage of working outwhich attributes to measure. #hey understand the issues, they know the terminology and they will welcome beingconsulted. nternal focus groups with the sales staff will prove highly instructive. #his internally generatedinformation may be biased, but it will raise most of the general customer issues and is readily available at little cost.

    t is wise to cross check the internal views with a small number of depth interviews with customers. -alf a do7enmay be all that is re!uired.

    When all this work has been completed a list of attributes can be selected for rating.

    About price

    2arket price#otal cost of use8alue for money

    About the company

    3eputation of the company1ase of doing businessnvoice claritynvoices on time

    Aboutstaff and service

    +ourtesy from sales staff3epresentative9s availability3epresentative9s knowledge3eliability of returning calls$riendliness of the sales staff+omplaint resolution3esponsiveness to en!uiriesAfter sales service#echnical service

    About delivery

    0elivery on timepeed of delivery

    About the product

    :uality of the productength of life of the product0esign of the product+onsistency of !uality3ange of products;rocessibility of the product

  • 7/26/2019 Lllll Llllo Ooooooo

    43/176

    -W -/0 #-1

  • 7/26/2019 Lllll Llllo Ooooooo

    44/176

    When planning the fieldwork, there is likely to be a debate as to whether the interview should be carried out withoutdisclosing the identify of the sponsor. f the !uestions in the survey are about a particular company or product, it isobvious that the identity has to be disclosed. When the survey is carried out by phone or face to face, co%operationis helped if an advance letter is sent out e'plaining the purpose of the research. ogistically this may not be possiblein which case the e'planation for the survey would be built into the introductory script of the interviewer.

    f the survey covers a number of competing brands, disclosure of the research sponsor will bias the response. f theinterview is carried out anonymously, without disclosing the sponsor, bias will result through a considerably reducedstrike rate or guarded responses. #he interviewer, e'plaining at the outset of the interview that the sponsor will bedisclosed at the end of the interview, usually overcomes this.

    -W -/0 A#$A+#< "1 21A/310?

    +ustomers e'press their satisfaction in many ways. When they are satisfied, they mostly say nothing but returnagain and again to buy or use more. When asked how they feel about a company or its products in open%ended!uestioning they respond with anecdotes and may use terminology such as delighted, e'tremely satisfied, verydissatisfied etc. +ollecting the motleys variety of adectives together from open ended responses would beproblematical in a large survey. #o overcome this problem market researchers ask people to describe a companyusing verbal or numeric scales with words that measure attitudes.

    ;eople are used to the concept of rating things with numerical scores and these can work well in surveys. nce therespondent has been given the anchors of the scale, they can readily give a number to e'press their level ofsatisfaction. #ypically, scales of , @ or B are used where the lowest figure indicates e'treme dissatisfaction and thehighest shows e'treme satisfaction. #he stem of the scale is usually !uite short since a scale of up to BB wouldprove too demanding for rating the do7ens of specific issues that are often on the !uestionnaire.

    2easuring satisfaction is only half the story. t is also necessary to determine customers* e'pectations or theimportance they attach to the different attributes, otherwise resources could be spent raising satisfaction levels ofthings that do not matter. #he measurement of e'pectations or importance is more difficult than the measurement ofsatisfaction. 2any people do not know or cannot admit, even to themselves, what is important. +an believesomeone who says they bought a ;orsche for its 4engineering e'cellence5? +onsumers do not spend their timerationalising why they do things, their views change and they may not be able to easily communicate or admit to thecomple' issues in the buying argument.

    #he same interval scales of words or numbers are often used to measure importance , @ or B being veryimportant and being not at all important. -owever, most of the issues being researched are of some importance forotherwise they would not be considered in the study. As a result, the mean scores on importance may show littledifferentiation between the vital issues such as product !uality, price and delivery and the nice to have factors suchas knowledgeable representatives and long opening hours. 3anking can indicate the importance of a small list of upto si' or seven factors but respondents struggle to place things in rank order once the first four or five are out of theway. t would not work for determining the importance of CB attributes.

    As a check against factors that are given a 4stated importance5 score, researchers can statistically calculate (or4derive5) the importance of the same issues. 0erived importance is calculated by correlating the satisfaction levelsof each attribute with the overall level of satisfaction. Where there is a high link or correlation with an attribute, itcan be inferred that the attribute is driving customer satisfaction. 0eriving the importance of attributes can show the

    greater influence of softer issues such as the friendliness of the staff or the power of the brand things that peoplesomehow cannot rationalise or admit to in a 4stated5 answer.

  • 7/26/2019 Lllll Llllo Ooooooo

    45/176

    W-A# 0 #-1 21A/3121

  • 7/26/2019 Lllll Llllo Ooooooo

    46/176

    #his raises the interesting !uestion what is achievable and how far can we go in the pursuit of customersatisfaction. Abraham incoln*s !uote about fooling people could be usefully modified for customer satisfactionresearch to read 4=ou can satisfy all the people some of the time, and some of the people all the time, but you cannotsatisfy all the people all the time5. As marketers we know that we must segment our customer base. t is no goodtrying to satisfy everyone, as we do not aim our products at everyone. What matters is that we achieve high scoresof satisfaction in those segments in which we play. btaining scores of I or B from around a half to two thirds oftargeted customers on issues that are important to them should be the aim. ;lotting the customer satisfaction scoresagainst the importance score will show where the strengths and weaknesses lie, (see diagram E) with the mainobective to move all issues to the top right bo'.

    0iagram E6 (" gra)h to sho! !here customer satisfaction needs to im)rove

    -W # /1 +/#213 A#$A+#< #/01 # >31A#1# 1$$1+#

  • 7/26/2019 Lllll Llllo Ooooooo

    47/176

    methods introduced by >ordon "ethune, chairman and chief e'ecutive since IIG, is a NF cash bonus paid toemployees each month if +ontinental is among the most punctual carriers. t makes sense to reward, in ;avlovianfashion, immediately after the event and not si' or twelve months down the line when the effect will have beenforgotten.

    #wo companies, both ostensibly committed to customer satisfaction, but one markedly outperforming the other. #hecustomer satisfaction scores are only part of the story.

    A customer satisfaction inde' is a snapshot at a point in time. ;eople*s views change continuously and theperformance of companies in delivering customer satisfaction is also changing. 2easuring satisfaction must be acontinuous process. #racking surveys provide benchmarks of one*s own company*s performance and, if competitorsuppliers are also being measured, there will be measurements of relative performance. #his places considerableonus on the researcher to design a survey that will accurately show real differences, one survey to another. #he!uestionnaire needs to be consistent so there is no dispute about answers differing because of changes to !uestions.#he sample of each survey must be large enough to provide a reliable base and the selection of the sample mustmirror earlier surveys so like is being compared with like.

    "enchmarking in customer satisfaction can go beyond comparisons with direct competitors. ome firms have takenthis type of benchmarking a step further. nstead of ust developing a benchmark on competitors, they identify thebest firm in any industry at a particular activity. .. "ean may be benchmarked for telephone order processing or

    customer service. American 1'press may be benchmarked for billing and payment transactions.

    #here has been considerable research into the links between customer satisfaction and employee satisfaction Oaplan M

  • 7/26/2019 Lllll Llllo Ooooooo

    48/176

    0181; A< A+#< ;A< #-A# 31+#$1 W1AO

  • 7/26/2019 Lllll Llllo Ooooooo

    49/176

    !uality assurance, plus research and development. "ecause each department evaluates suppliers differently, thecustomer satisfaction programme will need to cover the multiple views.

    #he adage in market research that we turn to again and again is the need to ask the right !uestion of the right person.$inding that person in customer satisfaction research may re!uire a compromise with a focus on one person % the keydecision maker& perhaps the transport manager in the e'ample of the trucks. f money and time permit, differentpeople could be interviewed and this may involve different interviewing methods and different !uestions.

    #he traditional first in line customer is an obvious candidate for measuring customer satisfaction. "ut what aboutother people in the channel to market? f the products are sold through intermediaries, we are even further from ourcustomers. A good customer satisfaction program will include at least the most important of these types of channelcustomers, perhaps the wholesalers as well as the final consumers.

    ne of the greatest headaches in the organisation of a business to business customer satisfaction survey is thecompilation of the sample frame the list from which the sample of respondents is selected. "uilding an accurate,up%to%date list of customers, with telephone numbers and contact details is nearly always a challenge. #he list heldby the accounts department may not have the contact details of the people making the purchasing decision. argebusinesses may have regionally autonomous units and there may be some fiefdom that says it doesn*t want itscustomers pestered by market researchers. #he sales teams* +hristmas card lists may well be the best lists of all butthey are kept close to the chest of each sales person and not held on a central server. "uilding a good sample frame

    nearly always takes longer than was planned but it is the foundation of a good customer satisfaction survey.

    +ustomer satisfaction surveys are often ust that surveys of customers without consideration of the views of lost orpotential customers. apsed customers may have stories to tell about service issues while potential customers are agood source of benchmark data on the competition. f a survey is to embrace non%customers, the compilation of thesample frame is even more difficult. #he !uality of these sample frames influences the results more than any otherfactor since they are usually outside the researchers* control. #he !uestionnaire design and interpretation are withinthe control of the researchers and these are subects where they will have considerable e'perience.

  • 7/26/2019 Lllll Llllo Ooooooo

    50/176

    W-A# -/0 "1 21A/310?

    n customer satisfaction research we seek the views of respondents on a variety of issues that will show how thecompany is performing and how it can improve. #his understanding is obtained at a high level (4how satisfied areyou the A"+ td overall?5) and at a very specific level (4how satisfied are you with the clarity of invoices?5).

    -igh level issues are included in most customer satisfaction surveys and they could be captured by !uestions suchas6

    What is your overall satisfaction with A"+ td?-ow likely or unlikely are you to buy from A"+ td again?-ow likely or unlikely would you be to recommend A"+ td to a friend or colleague?

    t is at the more specific level of !uestioning that things become more difficult. ome issues are of obviousimportance and every supplier is e'pected to perform to a minimum acceptable level on them. #hese are thehygiene factors. f a company fails on any of these issues they would !uickly lose market share or go out ofbusiness. An airline must offer safety but the level of in%flight service is a variable. #hese variables such as in%flightservice are often the issues that differentiate companies and create the satisfaction or dissatisfaction.

    Working out what !uestions to ask at a detailed level means seeing the world from the customers* points of view.

    What do they consider important? #hese factors or attributes will differ from company to company and there couldbe a long list. #hey could include the following6

    About delivery

    0elivery on timepeed of delivery

    #he list is not e'haustive by any means. #here is no mention above of environmental issues, sales literature,fre!uency of representatives* calls or packaging. 1ven though the attributes are deemed specific, it is not entirelyclear what is meant by 4product !uality5 or 4ease of doing business5. +ryptic labels that summarise specific issueshave to be carefully chosen for otherwise it will be impossible to interpret the results.

    +ustomer facing staff in the research%sponsoring organisation will be able to help at the early stage of working outwhich attributes to measure. #hey understand the issues, they know the terminology and they will welcome beingconsulted. nternal focus groups with the sales staff will prove highly instructive. #his internally generatedinformation may be biased, but it will raise most of the general customer issues and is readily available at little cost.

    t is wise to cross check the internal views with a small number of depth interviews with customers. -alf a do7enmay be all that is re!uired.

    When all this work has been completed a list of attributes can be selected for rating.

    About price

    2arket price#otal cost of use8alue for money

    About the company

    3eputation of the company1ase of doing businessnvoice claritynvoices on time

    Aboutstaff and service

    +ourtesy from sales staff3epresentative9s availability3epresentative9s knowledge3eliability of returning calls$riendliness of the sales staff+omplaint resolution3esponsiveness


OOOOOOOOOOO OOOOOOOOO - Life Hair Design · llllll llllll llllll llllll llllll llllll OOOOOOO OOOOOOO OOOOOOO OOOOOOO llllllllllllll SSSSSS SSSSSS SSSSSS SSSSSS SRSSSS SSSSSSSS SSSSSSSS

OOOOOOOOOOO OOOOOOOOO - Life Hair Design · llllll llllll llllll llllll llllll llllll OOOOOOO OOOOOOO OOOOOOO OOOOOOO llllllllllllll SSSSSS SSSSSS SSSSSS SSSSSS SRSSSS SSSSSSSS SSSSSSSS

1111111111111111 IIIIII IIIII 11111 1111111111 11111 lllll ...€¦ · 1111111111111111 IIIIII IIIII 11111 1111111111 11111 lllll lllll lllll lllll 111111111111111 11111111 US 20050014467Al

1111111111111111 IIIIII IIIII 11111 1111111111 11111 lllll ...€¦ · 1111111111111111 IIIIII IIIII 11111 1111111111 11111 lllll lllll lllll lllll 111111111111111 11111111 US 20050014467Al

I llllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll ......I llllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111

I llllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll ......I llllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111

Car All Lllll Lllll Lllll

Car All Lllll Lllll Lllll

United States Patent Application Publication Pub. No.: US ......I lllll llllllll II llllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111111111111111111 US 20030219006Al

United States Patent Application Publication Pub. No.: US ......I lllll llllllll II llllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111111111111111111 US 20030219006Al

I llllll lllll lllll llll llllll lllll lllll llllI llllll lllll lllll llll llllll lllll lllll lllll llll /111 *495.0001* 506 ATTORNEY GENERAL'S OPINIONS [VOLUME 57 . Opinion No. 74-68-Nove.mber

I llllll lllll lllll llll llllll lllll lllll llllI llllll lllll lllll llll llllll lllll lllll lllll llll /111 *495.0001* 506 ATTORNEY GENERAL'S OPINIONS [VOLUME 57 . Opinion No. 74-68-Nove.mber

Ne Ooooooo Oooooooooo

Ne Ooooooo Oooooooooo

I llllllll1111 lllll lllll Ill llllllll Ill lllll ... I I llllllll1111 lllll lllll Ill llllllll Ill lllll lllll11111l /I Ill1 LEAD SHEET 01 0969700' ~ - - - - - - - __ . SPACE ABOVE

I llllllll1111 lllll lllll Ill llllllll Ill lllll ... I I llllllll1111 lllll lllll Ill llllllll Ill lllll lllll11111l /I Ill1 LEAD SHEET 01 0969700' ~ - - - - - - - __ . SPACE ABOVE

I lllllll llll llill lllll lllll lllll lllll llll llll ... Heport t.\D==-A244 009 I lllllll llll llill lllll lllll lllll lllll llll llll USATHAMA U.S. Army Toxic and Hazardous Materials

I lllllll llll llill lllll lllll lllll lllll llll llll ... Heport t.\D==-A244 009 I lllllll llll llill lllll lllll lllll lllll llll llll USATHAMA U.S. Army Toxic and Hazardous Materials

lllll~l llllllllll ll llllllllll lllll lllll ...

lllll~l llllllllll ll llllllllll lllll lllll ...

I lllll llllllll Ill lllll lllll lllll lllll lllll lllll lllll lllll llllll Ill lllll llllI lllll llllllll Ill lllll lllll lllll lllll lllll lllll lllll lllll llllll Ill lllll llll

I lllll llllllll Ill lllll lllll lllll lllll lllll lllll lllll lllll llllll Ill lllll llllI lllll llllllll Ill lllll lllll lllll lllll lllll lllll lllll lllll llllll Ill lllll llll

I IIII IIIIIII Ill lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111ieeemilestones.ethw.org/images/a/aa/Ref2-US5657420.pdf · 2016-09-30 · "DSP Chips Can Produce

I IIII IIIIIII Ill lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111ieeemilestones.ethw.org/images/a/aa/Ref2-US5657420.pdf · 2016-09-30 · "DSP Chips Can Produce

Wednesday Dinner Menu - Detroit Gun Clubdetroitgunclub.org/wp-content/uploads/2019/07/Wednesday...ooooooo oooooooo ooooooo oooooooo ooooooo oooooooo ooooooo Title Wednesday Dinner

Wednesday Dinner Menu - Detroit Gun Clubdetroitgunclub.org/wp-content/uploads/2019/07/Wednesday...ooooooo oooooooo ooooooo oooooooo ooooooo oooooooo ooooooo Title Wednesday Dinner

I llllll lllll lllll lllll lllll lllll lllll lllll lllll ... Holdings Letter to SEC re...LETTER/MISC LTR December 17, 2018 0 CED/CFD/CRMD/MRD/NTD . COVERSHEET ... Joseph Augustin L.

I llllll lllll lllll lllll lllll lllll lllll lllll lllll ... Holdings Letter to SEC re...LETTER/MISC LTR December 17, 2018 0 CED/CFD/CRMD/MRD/NTD . COVERSHEET ... Joseph Augustin L.

Hllll llllllll I11 lllll lllll lllll lllll lllll lllll lllll lllll

Hllll llllllll I11 lllll lllll lllll lllll lllll lllll lllll lllll

Hall Ooooooo

Hall Ooooooo

lllllRll lllll Ill 11111 lllll lllll lllll lllll llllllll · 2020. 9. 22. · lllllRll lllll Ill 11111 lllll lllll lllll lllll llllllll . LM096749 Geolcgxal Survey . ' Department

lllllRll lllll Ill 11111 lllll lllll lllll lllll llllllll · 2020. 9. 22. · lllllRll lllll Ill 11111 lllll lllll lllll lllll llllllll . LM096749 Geolcgxal Survey . ' Department

I llllll lllll lllll lllll lllll lllll lllll lllll lllll ... · I llllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111 110152018001346

I llllll lllll lllll lllll lllll lllll lllll lllll lllll ... · I llllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111 110152018001346

I llllll lllll lllll lllll lllll lllll llll llll · I llllll lllll lllll lllll lllll lllll llll llll *710997* REQUEST: Adoption of the 2015-2020 Six-Year Transportation Improvement

I llllll lllll lllll lllll lllll lllll llll llll · I llllll lllll lllll lllll lllll lllll llll llll *710997* REQUEST: Adoption of the 2015-2020 Six-Year Transportation Improvement

I lllll llllllll II llllll lllll lllll lllll lllll lllll lllll lllll ...authors.library.caltech.edu/60157/2/US20020074565.pdf · 2016. 2. 19. · other, but the dielectric shells

I lllll llllllll II llllll lllll lllll lllll lllll lllll lllll lllll ...authors.library.caltech.edu/60157/2/US20020074565.pdf · 2016. 2. 19. · other, but the dielectric shells