KPI Deployment Guide

EUROPEAN ORGANISATION FOR THE SAFETY OFAIR NAVIGATION

EUROCONTROL

EUROPEAN AIR TRAFFIC MANAGEMENT PROGRAMME

KPI Deployment Guide

AIM/AEP/S-LEV/0007

Edition : 0.1Edition Date : 12 Nov 2001Status : DraftClass : General Public

DOCUMENT IDENTIFICATION SHEET

DOCUMENT DESCRIPTION

Document TitleKPI Deployment Guide

EWP DELIVERABLE REFERENCE NUMBER

PROGRAMME REFERENCE INDEX EDITION : 0.1

AIM/AEP/S-LEV/0007 EDITION DATE : 12 Nov 2001Abstract

This guide presents main activities required to deploy a performance measurement system.

KeywordsKPI Indicator Activity TemplateChecklist Deployment Guide

CONTACT PERSON : Paul Bosman TEL : 3333 UNIT : AIM

DOCUMENT STATUS AND TYPE

STATUS CATEGORY CLASSIFICATIONWorking Draft o Executive Task o General Public þDraft þ Specialist Task o EATMP oProposed Issue o Lower Layer Task þ Restricted oReleased Issue o

INTERNAL REFERENCE NAME : AHEAD Electronic Filing System

AIM/AEP/S-LEV/0007 KPI Deployment Guide

Page iv Draft Edition: 0.1

DOCUMENT APPROVAL

The following table identifies all management authorities that have successively approved thepresent issue of this document.

AUTHORITY NAME AND SIGNATURE DATE

Author/EditorErtan Ozkan 12 Nov 2001

ProgrammeManager

Conrad Cleasby 12 Nov 2001

Quality Assurance

KPI Deployment Guide AIM/AEP/S-LEV/0007

Edition: 0.1 Draft Page v

DOCUMENT CHANGE RECORD

The following table records the complete history of the successive editions of the presentdocument.

EDITION DATE REASON FOR CHANGESECTIONS

PAGESAFFECTED

0.1 12 Nov 2001 Creation All


Page vi Draft Edition: 0.1

- Page left intentionally blank -


Edition: 0.1 Draft Page vii

TABLE OF CONTENTS

1. INTRODUCTION ......................................................................................................1

1.1 Purpose and scope ................................................................................................1

1.2 References..............................................................................................................1

2. IMPROVEMENT MODEL .........................................................................................3

3. ACTIVITIES..............................................................................................................4

3.1 KPI Definition ..........................................................................................................4

3.1.1 Activity: Define KPIs....................................................................................4

3.1.2 Activity: Identify Production Process ...........................................................5

3.1.3 Activity: Brainstorming on KPIs ...................................................................6

3.1.4 Activity : Develop Data Collection Plan .......................................................7

3.2 Monitoring and Analysing Data .............................................................................8

3.2.1 Activity: Monitor KPI Compliance ................................................................8

3.2.2 Activity: Gather and Analyse Data...............................................................9

3.3 Activity: Improve Component..............................................................................10

3.4 Activity: Refine KPIs.............................................................................................12

4. CONCLUSIONS AND FUTURE WORK.................................................................13

APPENDIX A KPI DESIGN FORM..................................................................................15

APPENDIX B MAJOR PITFALLS...................................................................................16

APPENDIX C KPI CHECKLIST ......................................................................................17

APPENDIX D CUSTOMER FORMS................................................................................20

D.1 Who Are Our Customers?....................................................................................20

D.2 Customer Segments.............................................................................................22

D.3 Customer Evaluation Form..................................................................................23

D.4 Customer Background Information.....................................................................24

APPENDIX E IMPROVEMENT FORM............................................................................25

E.1 Improvement Matrix..............................................................................................25

E.2 Improvement Plan ................................................................................................26

E.3 Improvement Selection Checklist........................................................................27

APPENDIX F DATA COLLECTION PLAN .....................................................................28


Edition: 0.1 Draft Page 1

1. INTRODUCTION

1.1 Purpose and scope

This document presents the general framework for the deployment of KeyPerformance Indicators (KPIs) within the context of a quality management system.

It contains activities derived from a simple 3-tier improvement model. Activities aregiven in a step-by-step manner in order to give a clear picture what is required. Eachactivity is given in the following template:

Purpose:Purpose of the activitySteps:Steps needed to carry out the activityInput Artefacts:Required Inputs

Resulting Artefacts:Created outputs

Frequency:Frequency of the activityRole:Who is responsible for this activityTips and Tricks:Additional comments, tips and tricks

Each activity contains references to the applicable guides and required documentseither in its input artefacts and/or its steps. If necessary, the activity is supported bya flowchart. Appendices contain some useful information that has not been coveredin the guides.

The document is organised as follows:

• Chapter 2 presents 3-tier improvement model and basic loop in aperformance measurement system.

• Chapter 3 gives activities required to deploy a performance measurementsystem.

• Chapter 4 presents conclusions and future work

Throughout this document the term “performance measurement system” is used todenote the decision support system that contains a set of Key PerformanceIndicators (KPIs) with well-defined measurement, monitoring, analysis andimprovement procedures.

1.2 References

[1] Introduction to KPI Models, Edition 0.1, AIM/AEP/S-LEV/00004, 29 Oct 2001.

[2] KPI Portfolio, Edition 0.4, AIM/AEP/S-LEV/0005, 23 Oct 2001.

[3] KPI Top-12, Edition 0.1, AIM/AEP/S-LEV/0006, 13 Nov 2001.


Page 2 Draft Edition: 0.1

[4] KPI Measurement, Monitoring and Analysis Guide, Edition 0.1, AIM/AEP/S-LEV/0008, 30 Nov 2001.

[5] Change Management the 5-step Action Kit, C. Rye, ISBN 0749433809, KoganPage Limited, 2001.

[6] Major Pitfalls of Performance Measurement Systems - Module V, Randy R.LaBarge, Pacific Northwest National Laboratory, U.S. Department of Energy, PBM-SIG Fall ‘99 Meeting, 17 November 1999, Washington D.C.

[7] Handbook for Basic Process Improvement, May 1996,http://www.balancedscorecard.org/files/bpihndbk.pdf.

[8] The Process Improvement Notebook, Department of the Navy, Total QualityLeadership Office, Publication No: 97-01, 1997.

[9] Operational Performance Measurement Increasing Total Productivity, W.Kaydos, ISBN 1574440993, CRC Press LLC, 1999.



2. IMPROVEMENT MODEL

The main objective of a performance measurement system is to provide a basis (i.e.,facts) for continual improvement:

Figure 1. 3-Tier Improvement Model

The above improvement model summarises basic skills required to deploy aperformance measurement system. At the lower layer technological know-how isrequired to support data collection and distribute data to the related persons. In thesecond layer is where KPIs are defined, collected, monitored and analysed. The toplayer, simply the objective, is to improve the business (i.e., product, process orsystem).

However this is not a one-shot approach. There is a continuous cyclical process.This cycle focuses on the internal processes (diagnostic measures) and externaloutcomes (strategic measures). The system's control is based on KPIs that aretracked continuously over time to look for trends, best and worst practices and areasfor improvement.

Figure 2. Loop in a Performance Measurement System

Define KPIs

Monitor KPICompliance

Gather and AnalyseData

Refine the KPIS

ImproveProduct/Service,Process, System

Technological Know-how(Databases, Intranet, O/S, etc.)

Performance Measurement System(Define KPIs, Collect and Analyse Data)

Business Improvement(Achieve the Objective)



3. ACTIVITIES

3.1 KPI Definition

3.1.1 Activity: Define KPIs

Purpose:

To generate a possible set of KPIs covering system, customer satisfaction, human factors,product/services and processes.

Steps:

1. [Optional] Create the list of products produced and/or services provided by your organisation2. [Optional] Identify your major (segments of) customers and their profiles (i.e., services,

products, delivery methods and requirements)3. [Optional] Identify your system (organisational) requirements and goals4. [Optional] Identify list of principal processes used in production and service provision via the

Activity Identify Production Process.5. Create a possible list of KPIs for the following categories: financial, customer satisfaction,

human factors and business processes via the Activity Brainstorming on KPIs.6. Formulate each KPI by using the KPI Design Form as a result of brainstorming meeting.7. Apply KPI Checklist on the list of possible KPIs (result of the Activity Brainstorming on KPIs)

in order to select your initial set of KPIs.8. Refine data collection methods for each KPI via the activity Develop Data Collection Plan.9. [Optional] Set an objective for each selected KPI (if you have historical data)Input Artefacts:

None

Resulting Artefacts:

• List of Selected KPIs

Frequency:

Once initially and after a change in the system, products, processes, etc.

Role:

Quality manager

Tips and Tricks:• Steps between 1 and 4 are optional prerequisites of this activity. They are given in order to

illustrate the complete picture. Ideally implementation should begin with identifying externalcustomer segments and determining what they want. Then a strategy should be developedwhich will lead to defining the organisation’s key deliverables, KPIs and productionprocesses. KPIs should then be implemented in the principal processes according to wherethe greatest return or risk exists. Though this sequence is conceptually sound it is not alwaysnecessary to have a well-defined strategy before implementing KPIs. Since everybody wantswhatever they are buying cheaper, better and faster. There are usually significant areas ofopportunity available that can be addresses with KPIs while any strategic questions are beingresolved [8].

• For Step 2 you may use the following forms:o Who are our customers?o Customer Background Informationo Customer Evaluation Formo Customer Segments



3.1.2 Activity: Identify Production Process

Purpose:

To chart the current activities involved in achieving a particular service or product and theirinterrelationship and identify all of the component parts of the current process and commit each toa 'Post-it'.

Steps:

1. Establish the dependencies of each component to others and move 'Post-its' to define logicalsequences of events.

2. Establish component activities or sequences which can parallel other components orsequences and move 'Post-its' to illustrate these parallel relationships.

3. Add lines and arrow connections to connect activities and sequences and illustrate the flow ofactivity.

4. [Optional] For clarity, realign 'Post-its' to place the 'primary' process (the one to which otheractivities and parallel processes contribute) in a straight line across or down the overall chart.

5. [Optional] Add to each component or sequence the name of the party responsible for itscompletion. This may help to identify inefficiencies resulting from 'hand-offs' between parties.

Input Artefacts:

None


• Chart of Production Processes

Frequency:

Before process improvement and selecting KPIs for the process

Role:

Quality manager and responsible person(s) for this process

Tips and Tricks:

• Make sure that each (human) business actor express a role, not a person!• This technique is frequently used during Business Process Re-engineering to establish the

current situation. It could also be used to illustrate a proposed new process prior during theplanning and implementation stage of a change situation. Process mapping is essentially aform of flowcharting and users may wish to increase clarity by using proper flowchartsymbols.



3.1.3 Activity: Brainstorming on KPIs

Purpose:

Generating large numbers of ideas around the KPI theme from a group of people in a short periodof time.

Steps:

1. Gather an appropriate group together. Typically, this will comprise those with a vestedinterest in the KPIs, that is, those involved with or impacted by it.

2. Advise the group of the problem or issue and check that all participants fully understand theconcept of the KPI.

3. If appropriate break the issue down into manageable topics, eg 'the deployment of KPIs 'might split into 'how to generate the right KPI specifications', 'how to measure KPIs’, 'how toreport KPIs' and 'how to interpret KPIs and use for improvement’. State the sub-issue whichthe group are to tackle first and note it at the top of a flipchart page.

4. Invite the members of the group to call out their suggestions to resolve the problem or issue.Record every idea, as stated, on the flipchart so that the developing list is visible to the group.Encourage succinct statements.

5. Do not allow discussion or criticism and aim for quantity not quality.6. Permit laughter at wacky ideas – they often have tremendous potential – but quash ridicule of

group members when they present ideas which may seem outrageous.

Input Artefacts:

• Introduction to KPI Models• KPI Portfolio• KPI Top 12


• List of possible KPIs

Frequency:

Before designing the KPIs.

Role:

Brainstorming meeting is organised by the quality manager

Tips and Tricks:

• Brainstorming is a powerful, versatile and simple technique for generating ideas but demandsdisciplined facilitation by the group leader (who should not participate in the ideas generationprocess itself). It can be applied to almost any problem.



3.1.4 Activity : Develop Data Collection Plan

Purpose:

Determination of the tracking of the whereabouts of specific 'events'. Typically these events areproduct defects or problem areas within a process and/or a system and It enables their locationand frequency of occurrence to be established.

Steps:

1. Identify the 'event' to be tracked.2. Determine a means of data collection. Examples might be:

• A plan of the layout of the area in which the 'event' is occurring;• An illustration of the product incurring the 'event' (defect);• An example of the product itself;• A process-map of the process within which the 'event' is occurring;• A checklist listing the zones within the overall organisation in which the 'event' is

occurring.3. Determine who is best placed to collect the 'event' data.

Input Artefacts:

• KPI for which data to be collected


• KPI Data Collection Plan

Frequency:

For each selected KPI

Role:

Quality manager and/or responsible person(s) where the event is occurring (i.e., a process,system, etc.)

Tips and Tricks:

• The accuracy of data collection is paramount while determining data collection method andcare should be exercised when determining how, and by whom, data is to be collectedparticularly when large geographical areas are under examination.

• For production processes, charts of production processes created in the activity IdentifyProduction Process helps the team determine who should collect data and where in theprocess data should be collected.

• Use the form Data Collection Plan in order to formalise your work



3.2 Monitoring and Analysing Data

3.2.1 Activity: Monitor KPI Compliance

Purpose:

Collect data for the KPIs as stated in the KPI Data Collection Plans and monitor KPI compliance.This activity is the part of the operational control.

Steps:

1. Record the event data as appropriate –stated in the KPI Data Collection Plans- until suchtime as an adequate sample has been obtained.

2. [Optional] If there is no objective set for this KPI and derive the KPI objective.3. Check whether the KPI objective is attained.4. If the objective is not attained, evaluate results to determine action to resolve the root cause

of the problem.

Input Artefacts:

• KPI Data Collection Plans• KPI Measurement, Monitoring and Analysis

Guide


• Operational Monitoring Reports

Frequency:

Depending the critically and stability of the operations. Usually daily or weekly.

Role:

Operational Manager

Tips and Tricks:

• Data sample size is also critical. As a general rule, if in doubt increase the sample size.



3.2.2 Activity: Gather and Analyse Data

Purpose:

Gather KPI measurements and analyse them.

Steps:

1. Gather KPI measurements.2. Create executive reports containing vital information about the performance3. Check overall KPI compliance.4. Analyse each KPI and determine problematic components (a component may be one of the

following: product, service, process, system, resource or expansion of market)5. Check whether an improvement is required and possible for problematic components.6. Prioritise and select the component(s) to be improved and execute the activity Improve

Component for the selected component(s)Input Artefacts:

• KPI Measurement, Monitoring and AnalysisGuide


• Executive Reports• Selected Components for Improvement

Frequency:

Depending the critically and stability of the operations. Usually monthly, quarterly or annually.

Role:

Quality Manager

Tips and Tricks:

• For step 4-6 you may use the form Improvement Matrix to analyse improvement alternatives.• For step you may use the form Improvement Selection Checklist to verify whether a correct

component is selected.



3.3 Activity: Improve Component

Purpose:

Improve a selected component. Component can be one the following: product, service, process,system, resource or expansion of market.

Steps:

1. Establish a well-defined improvement objective.2. Organise a team to improve the component. This involves selecting the right people to serve

on the team; identifying the resources available for the improvement effort, such as people,time, money and materials; setting reporting requirements; and determining the team’s levelof authority.

3. Identify the root causes that prevent the component from meeting the objective. The teambegins the Plan-Do-Check-Act Cycle here, using the cause-and-effect diagram orbrainstorming tools to generate possible reasons why the component fails to meet thedesired objective.

4. Develop an improvement plan for implementing a change based on the possible reasons forthe component’s inability to meet the objective set for it. These root causes were identified inStep 3.

5. [Optional] Check whether KPIs related to the component do exist and they provide all therequired data for the evaluation of the improvement objective. If not, define new KPI(s) forthis component and collect baseline data.

6. Test the changed component and collect data.7. Assess whether the changed component is stable. The team uses a control chart or run chart

to determine stability. If the component is stable, the team can move on to Step 8; if not, theteam must return the component to its former state and plan another change.

8. Assess whether the change improved the component. Using the data collected in Step 6 anda histogram, the team determines whether the component is closer to meeting the processimprovement objective established in Step 1. If the objective is met, the team can progress toStep 9; if not, the team must decide whether to keep or discard the change.

9. Determine whether additional improvements are feasible. If yes, re-start the activity from Step3.

10. Standardise the change and reduce the frequency of data collectionInput Artifacts:

• Selected Component for Improvement• List of KPIs related to the component

Resulting Artifacts:

• Improved component

Frequency:

For each improvement

Role:

Quality Manager

Tips and Tricks:

• You may use the form Improvement Plan in order to document your improvement effort.



Figure 3. Flowchart of Activity Improve

No

Yes

Yes

No

Yes

No

Yes

Yes

Step 9More Improvement?

Step 1Establish the improvement

objective

Step 2Organise the right team

Step 6Test the change and collect data

Step 3Identify root causes for lack of capability

Step 4Plan to implement the change

Step 5Modify the KPIs (if necessary)

Step 8aImproved?

Step 7Stable?

Step 10Standardise the change and reduce

the frequency of data collection

No

Step 8bKeep?



3.4 Activity: Refine KPIs

Purpose:

Handles changes in KPIs (i.e., addition, modification, deletion). Sets or modifies KPI objectives.Steps:

1. [Optional] Add missing KPIs for critical components. Apply KPI Checklist for the new KPIs.Determine data collection methods for each KPI via the activity Develop Data Collection Plan

2. [Optional] If necessary, refine data collection methods for each KPI via the activity DevelopData Collection Plan after a change in the associated component.

3. [Optional] Delete KPIs that are not providing required feedback.4. [Optional] Set/Modify KPI objectives.

Input Artefacts:

• List of current KPIs• KPI Data Collection Plans


• Modified KPI list• Modified KPI Data Collection Plans

Frequency:

Not applicable

Role:

Quality Manager

Tips and Tricks:



4. CONCLUSIONS AND FUTURE WORK

This document presents general framework for a performance measurementsystem. The ultimate goal of the performance measurement is to ensure customersatisfaction by providing high quality services/products in a cost effective manner.The deployment of a performance system can be quite costly depending on thenumber of Key Performance Indicators (KPIs) and the effort required to measurethem. Therefore selecting the appropriate KPIs is the key for successful applicationof this framework and common sense must prevail when selecting KPIs.

The framework and forms associated with it are proposals. Therefore it is likely thatthis document will evolve with your feedback.



- Page left intentionally blank -



APPENDIX A KPI DESIGN FORM

You may use the following form to design your KPIs:

KPI DESIGN FORM

KPI Name <Name of the KPI>

Description <Description of the KPI>

Objective <Objective of the KPI>

Type o Quantitative o QualitativeEffort o Low o Medium o HighUnit <Unit of the KPI>AssessmentMethod

<How to measure the KPI>

PossibleTools

<Name of possible tools which can be used in the KPI measurementand analysis>

o Day o Week o MonthAnalysisFrequency o Quarter o Year o Year+Comments <Additional Comments>



APPENDIX B MAJOR PITFALLS

Major Pitfalls of Performance Measurement Systems are given in the [6]:

§ Measures not linked to strategy - Critical to do initially, but also revisitwhen either the organisational strategy or structure changes

§ Measures not driven into organisation(s) below: Breaks the linkagewith overall strategy. Should be driven into staff performance agreementsat all appropriate levels.

§ Too many measures: Creates lack of focus on what is really critical tomanaging your business.

§ Not enough critical measures: You could be missing information vital tooperations.

§ Focusing only on the short-term: A cross-section of past (lagging),present and future measures is critical.

§ Conflicting measures: Sub-optimises staff or organisationalperformance. Example: Measuring reduction of office space per staffmember while also measuring staff satisfaction with facilities.

§ Measuring progress too often: Could result in unnecessary effort andexcessive costs, resulting in little or no added value

§ Not measuring progress often enough: May not know about potentialproblems until it is too late to resolve easily.

§ Collecting too much data: Could result in a mountain of data that reallydoesn’t tell us anything more than a lesser amount of the same data.

§ Collecting inconsistent, unrepresentative or unnecessary data:Critical to understand what the data will look like, when it will be collected,at what frequency, by whom and what it means, up front.

§ Reducing the value of data: Too much data roll-up (summary) can maskthe impact of potentially significant events or trends

§ Driving the wrong performance: Be careful that the measure(s) youselect will result in the desired result.

§ Encouraging competition and discouraging teamwork: Measuringvertically frequently pits one internal organisation against the others. Try tomeasure horizontally.

§ Failure to base business decisions on data: Developing performancemeasures or collecting data only to comply with a requirement doesnothing to improve the position of the company.



APPENDIX C KPI CHECKLIST

Here is the KPI checklist you may use to select each KPI:

KPI CHECKLIST

KPI : <KPI Name> Date: <dd/mm/yyyy>

Place a checkmark next to all of the items that apply to your KPI.

o Does the KPI motivate the right behaviour?o Is the KPI measurable?o Is the measurement of this KPI affordable (cost-effective)?o Is the target value attainable?o Are the factors affecting this KPI controllable by you?o Is the KPI meaningful?

Note: If you have selected an appropriate KPI, you should be able to check all of the items above.

The selection process is complicated by the number of potential KPIs and must betempered by considerations such as:

• Organisational experience with KPIs

• Type of behaviours to be motivated

• Cost and effort of collection.

Common sense must prevail when selecting KPIs. It should be remembered that thegoal is to ensure customer satisfaction by providing high quality services/products ina cost effective manner.

To meet above goals, AIS organisations should consider the following principleswhile selecting or defining new indicators:

• An indicator must motivate the right behaviour

• An KPI must be measurable

• An KPI must be affordable

• The objective set for an KPI must be attainable

• Factors effecting the indicator must be controllable by the service provider

• An KPIs must be meaningful to all the parties.

Indicators that motivate the right behaviour: When choosing KPIs, it must be firstfocussed on the behaviour that is wanted to be motivated. There are no hard andfast rules for this; first a KPI must reflect the performance at the right areas:



• Processes and factors most critical to business

• Processes and factors where most improvement needed

• The most dissatisfied client group

• The most politically powerful client group

Then it must motivate a pre-defined behaviour in these areas:

• To reducing costs and/or defects

• To increase production

• To speed time-to-market

• Etc.

An initial set of KPIs must be picked up in order to measure performance related tothese behaviours.

Measurable: If it is not affordable to measure something to represent a servicequality, that KPI is worthless and should not be considered. As an example user(i.e., end-to-end) response time is a very popular KPI among users. Certainly, this isone of the key factors shaping the users’ opinions about the quality of service theyare receiving. Unfortunately, measuring user response time on an end-to-end basisis still technical challenge today. At the other end of the feasibility spectrum is theavailability of a service. This is relatively straightforward and can be measured with aminimum effort and difficulty.

Affordable: No organisation has unlimited resources. The amount that can be spenton delivering any service is limited. Therefore, in setting objectives for a KPI, it isalso necessary to consider whether the desired objective (i.e., service level) isaffordable. This might also be thought of as being cost effective. An KPI should notrequire a heavy investment of time and money. In some cases, it will be necessaryto devise alternative KPIs if the required data is not easily obtainable.

Another important issue in this topic is the easiness of data collection. If KPIs cannotbe easily gathered, then they will quickly lose favour, and eventually be ignoredcompletely. No one is going to spend an excessive amount of time to collect KPIsmanually. Ideally, all KPIs will be captured automatically, in the background, withminimal overhead.

Attainable: To be useful, the KPIs must be set to reasonable, attainable objectives.It may be difficult to select an initial, appropriate setting for a KPI, especially whenthere is no available historical record of those KPIs. Organisations with active KPIprograms will have the data needed to set a proper baseline. Others will have toperform an initial assessment to establish that baseline. Unless strong historicalmeasurement data is available, it should be expected to re-visit and re-adjust thesetting at a future date. In addition a realistic tolerance level must be included whilesetting the objectives in the absence of historical measurement data.

Controllable: A KPI must represent something that is controllable. That is, theservice provider must have the ability to exercise control over factors that determine



the level of service delivered. The service provider has control over bringing an AISpublication to the post office, but has no control over the speed by which the postoffice delivers the mail. Thus, an objective that "all AIS publications will be deliveredto our clients within 48 hours after production completion" is unfair and likely to bede-motivating to the service provider.

Examples for this issues can be extended very easily: In a country where there arefrequent power failures and the budget of AIS organisation does not permit thepurchase of a standby generator to prevent service interruptions during thesefailures, strikes by union workers and poor service by a telecommunication companywithout a reasonable alternative. These are just a few of the factors that can placecertain objectives behind the control of the service provider.

Meaningful: In the context of the service level agreement a KPI must be meaningfuland understandable to all the parties (i.e., client and service provider). Another wayof stating this is to say that it must be relevant.

Lets take an AIS organisation, which provides some of its services/products viaInternet, as an example. Service provider decides to uses packet collisions anddropped packets in order to illustrate availability of its LAN component. They can beeasily captured however they mean little to anyone other than a network engineer orsystem administrator. Although these KPIs might impact on the availability from theclient’s perspective, the relevance of such KPIs to the service they receive is quitedifficult to grasp.



APPENDIX D CUSTOMER FORMS

D.1 Who Are Our Customers?

The ultimate goal of any improvement effort is to better satisfy the needs of thecustomer. Thus, before formulating any improvement goals, the organisation mustfirst assess who are their customers — who are they in business to support?Thinking about customers can raise fundamental questions about the purpose andmission of the organisation.

The following questions from Deming are useful to stimulate thinking aboutcustomers:

§ Who makes the decisions about whether to buy your product or service?

§ How do you distinguish between quality as your customer perceives it andquality as your managers and work force perceive it?

§ How does the quality of your product, as your customer sees it, agree withthe quality that you intended to give him/her?

§ Do your customers think that your product lives up to their expectations?

§ What do you know about the problems of your customers in the use ofyour products? What tests do you make of your products in service?

§ Do you depend on complaints from customers to learn what is wrong withyour product or service?

§ Are your customers satisfied with the service that you provide? If yes, whatis satisfactory about it? How do you know?

§ Will your customers of today be your customers a year hence? Two yearshence?



WHO ARE OUR CUSTOMERS?

Customer Products/Services Used



D.2 Customer Segments

Since most organisations have many customers, it is typical for organisations toseparate customers into smaller subgroups, often referred to as segments, using avariety of methods. Most organisations segment customers based on what productsand/or services they use. Customer segmentation assists in defining therequirements of different groups. It also provides a basis for understandingsimilarities among customers. It also clarifies which customers may be impacted byimprovement efforts focused on particular products or services.

CUSTOMER SEGMENTS

SEGMENTCUSTOMERS

SEGMENTCUSTOMERS

SEGMENTCUSTOMERS

SEGMENTCUSTOMERS

SEGMENTCUSTOMERS

SEGMENTCUSTOMERS



D.3 Customer Evaluation Form

This form should ideally be filled after customer interviews. However if interviewingcustomers is too costly or impossible in your schedule simply fill the form with yourperception.

CUSTOMER EVALUATION FORM

Date <dd/mm/yyyy>Name of the CustomerContact PersonTelFaxE-mailPostal Address

What products and services are currently provided to this customer?

What are the most important features or characteristics of the products/services provided to thiscustomer?

What are the major delivery format and delivery methods used for this customer?

What aspects of products/services is the customer satisfied with?

What needs improving?

Additional Comments and Observations



D.4 Customer Background Information

This form is to be used before customer interviews and/or evaluation.

CUSTOMER BACKGROUND INFORMATION

Date <dd/mm/yyyy>Name of the CustomerContact Person

What products and/or services has this customer acquired or used from your organisation in the past?

How often does this customer acquire products/services from us?

How long has this customer been using our products/services?

How much of our budget is related to products/services for this customer?

Does this customer have any pattern in the acquisition or use of our products/services?

Does any complaint data exist to help clarify customer requirements?

Do other customer satisfaction data exist?

Does this customer refer other organizations to you? Who?



APPENDIX E IMPROVEMENT FORM

E.1 Improvement Matrix

The improvement matrix is a tool for the initial and continuous analysis of thepossibilities a business has when attempting to maximise their competitive positionor advantage in the marketplace. Through an integrated brainstorm and analysis ofall the possibilities a business has for improvement regarding elements on the "Y"axis and their relationship to the elements on the "X" axis of this matrix, a companymay illustrate relationships of possibilities that may enhance their competitiveadvantage in the market. Possibilities for improvements / enhancements to businessbecome apparent when the range of possibilities is clearly illustrated; how can aconsumer or specialty product add value through service attachments or how can aservice provide the necessary products to sustain itself or further its image forcompetitive image in the marketplace... and so on, throughout the possibilities thatmay emerge through careful analysis of market potentials.

IMPROVEMENT MATRIX

Components to be ImprovedResources Processes/

SystemsProduct Expansion

of MarketService Etc.

Commodity

Industrial Product

Consumer Product

Speciality Product

Service/Industry

Service/Consumer

Service/Speciality

Etc.



E.2 Improvement Plan

IMPROVEMENT PLAN

Selected Component: <Name of Selected Component> Date: <dd/mm/yyyy>

Description <Briefly describe the component chosen for improvement. Use one form for eachcomponent. Note the reasons for selecting this component>o System o Process o ProductType of Component

to be Improved o Service o Resource o Expansion of Market

Improvement Goal(s)

<Describe the process improvement goal or desired change in the products or services associated with theimprovement effort>

Customer Impact

<Identify the desired customer impact of the improvement effort>

Products and Services Effected

<Describe the products/services that will be affected by improvement activities>

Resources

<Give required resources in order to achieve the improvement (human, training, financial, etc.)>

Reporting Requirements

<Clarify the improvement team’s reporting requirements: to whom do they report, how frequently and in whatformat>

Suggested Timeline

<Briefly describe schedule for the improvement>

Existing KPIsName Description

KPIs to DevelopName Description



E.3 Improvement Selection Checklist

This form may be used for each component that is considered for improvement.

IMPROVEMENT SELECTION CHECKLIST

Component : <Component Name> Date: <dd/mm/yyyy>

State problems of expectations identified for this component

1

2

3

4

Place a checkmark next to all of the items that apply to your process.

o The component and its associated improvement effort can be defined. (Be careful not to pick something toobig. It should be possible to complete the improvement effort within 90 days.)

o A problem in the component occurs frequently. (A Pareto analysis may be helpful.)o The problem area is well known and it is visible by the customer (external) or in the office (internal).o Improvement of this component is important to the customer or the business.o People will appreciate it if the component is improved.o There is a good chance of success in improving the component.o No one else is currently working on this component.o Required changes can be put into effect with little or no outside helpo This is truly an improvement effort, not just an attempt to impose a solution on a problem

Note: If you have selected an appropriate component, you should be able to check all of the items above.



APPENDIX F DATA COLLECTION PLAN

The Data Collection Plan summarises the important information about this criticaltask on to one form. Complete the form for each KPI for which data will be collected.It is likely that this form will be used many times.

DATA COLLECTION PLAN

KPI Name: <Name of KPI> Date: dd/mm/yyyy

What kind of data is to be collected?

<Describe the type of data that will be collected>

How will the data be collected?

<Specify how the data will be collected. Systematic procedures concerning the construction of a recording form,consistent use of the recording form, and data sampling must be formalized and described here>

When will the data be collected?

<Describe the appropriate time interval to be used in data collection in this section. This interval will bedetermined by the cycle time of the process>

Where will the data be collected?

<Describe where the data will be collected. All measurements must occur in the same location>

Who will collect the data?

< Record the names of the individuals who will carry out the specific data collection task>

KPI Deployment Guide

Documents

Transcript of KPI Deployment Guide