Impression

IMPRESSION 0103F07_KPI Manual_Part I

IST-2000-28760 of 14

IINNFFOO RRMMAATTIIOONN SS OOCCIIEETTII EESS TTEECCHHNNOO LLOO GGYY (( IISS TT)) PP RROO GGRRAAMMMM EE

IMPRESSION

“Initiative for Software Requirements Management Best Practice in Embedded Systems and E-Services Organisations”

DELIVERABLE EXTERNAL

WORKPACKAGE: WP2 – Implement systematic performance monitoring

D3-KPI Manual

Part I - General Methodology

Authors: C. Sorovou, D.Politou, A.Kalligeris, S.Topouzidou (ATC)

Submission Date: 5/10/2001

Partners: Athens Technology Center (GR), The City University (UK), University of Lancaster (UK). Members: Automate (GR), Integrated Aerospace Sciences Corporation (GR), Oxford Company (GR), International Carpet Center (GR), KOLIOS (GR), Euroelectronics (GR), Initiative Media (GR), PRISMA (GR), DISIOS (GR).


IST-2000-28760 of 14

CONTENTS 1. INTRODUCTION..............................................................................................................................................................3

1.1. DEFINITIONS, ACRONYMS AND ABBREVIATIONS....................................................................................................3 1.2. LIST OF RELATED DOCUMENTS..................................................................................................................................3

2. METHODOLOGY FOR DEFINING, DEVELOPING, USING AND MAINTENING KPIS .....................3 2.1. GENERAL PRINCIPLES..................................................................................................................................................3 2.2. DEVELOPMENT AND USE OF KPIS ..............................................................................................................................3 2.3. METHODOLOGY AT A GLANCE....................................................................................................................................3

3. APPENDIX A. INDICATIVE PROJECT TEAM S TRUCTURE........................................................................3


IST-2000-28760 of 14

1. INTRODUCTION

The purpose of the Key Performance Indicators (KPIs) Manual developed in the context of the IMPRESSION project is to assist the IMPRESSION members to identify, develop and measure KPIs, in order to acquire the benefits of best practice requirements management in software development for embedded systems & systems for e-business and e-services.

The manual is designed to be used by the IMPRESSION members as a practical guide to the process of introducing and using KPIs, following a methodology for their implementation.

This document describes the general methodology to be followed for defining, developing, measuring, evaluating and reporting KPIs. The Application of the IMPRESSION KPI Manual methodology for each user organisation can be found in the respective Part II documents.

1.1. Definitions, Acronyms and Abbreviations

KPI: Key Performance Indicator

CSF: Critical Success Factor

1.2. List of Related Documents

IST-2000-28760 IMPRESSION Project Annex I – Description of Work

D2- Yardsticks for each user

D3-KPI Manual Part II - Applying the KPI Manual

Ausindustries, “Key Performance Indicators Manual”, Pitman Publishing, Melbourne, 1996.


IST-2000-28760 of 14

2. METHODOLOGY FOR DEFINING, DEVELOPING, USING AND MAINTENING KPIs

This section provides an overview of the total process for developing and using KPIs. It describes the general principles regarding the KPIs as well as the eight steps to be followed in the process of development and use of them.

2.1. General Principles

Key Performance Indicators are the measures that monitor the performance of 'key result areas’ of business activities, which are absolutely critical to the success and growth of the business.

The development and use of KPIs should form the basis for the analysis of an organisation’s current performance, its future requirements and the improving strategies required for ongoing success.

Some general principles regarding the KPIs should be taken under consideration during their application:

• KPIs should not be an end in themselves, but be considered as an aid to management. They are a start to a proper informed debate that should lead to a plan for improvement.

• They should be seen within their local context and have more a meaning as a comparison over time than as a comparison between organisations.

• A set of performance indicators should be balanced. For example, measures of efficiency should be set against measures of effectiveness and measures of cost against quality and user perception.

• After being proposed and applied, KPIs should be reviewed and updated. The review determines the management utility of each indicator and the feasibility of getting source data for continuing use.

• The targeted performance description, which is described in measurable terms through the KPIs, must be deployed to the organisational level that has the authority and knowledge to take the necessary action.

Every initial proposal for KPIs is expected to be imperfect. However, it is important for the organisations to understand and apply the appropriate KPIs so that they can develop some experience in using them, from which they can derive real expertise.


IST-2000-28760 of 14

2.2. Development and use of KPIs

There is a nine steps approach in the procedure of development and use of KPIs, which is divided into four phases: Initiate, Build, Implement and Review.

PHASE I - INITIATE

STEP 1: Create the project team

It is proposed that a cross-functional team will be set up to implement the KPIs measurement. The team must be consisted of high- level employees, coming from different sections that have a clear view of the organisation’s goals and priorities. The team must also be familiar with the performance measures or be given a short training course, otherwise. The number of the team members depends on the size of the organisation and may vary between 4 to 6 employees. An indicative structure of the Project team is presented in Appendix A.

STEP 2: Align KPIs development with key areas and improvement strategies in requirements engineering

A strong correspondence must be established between the key areas, the Business Goals to be achieved and the improvement strategies to be followed, as KPIs will be developed to support these strategies. Therefore, the fist step is the definition of the key areas and the respective Business Goals, which are critical for the organisation, so that they can be aligned with the KPIs we intend to develop. Such Business Goals may be:

§ Improve RE Process Maturity § Improve Managerial Satisfaction § Improve User Satisfaction § Improve Efficiency of the System Implementation Process § Improve Maintainability of System § Improve Productivity of S/W development

The KPIs should focus on simple ideas that represent key goals of high relevance to the organisation in terms of the project assessed to ensure that the organisation comprehends the link between them and the business strategy.

STEP 3: Explain the purpose of the use and development of KPIs

The aim of measuring an organisation’s performance is to assess how well it is implementing strategies and action plans. To improve an organisation’s performance you must measure it first.

It is therefore necessary to develop KPIs, which is the set of indicators that will monitor the performance of 'key result areas’ of business activities. KPIs are a significant tool to assist improvement towards organisational objectives aiming to the


IST-2000-28760 of 14

overall prosperity. KPIs also create an essential feedback and learning mechanism to support management decisions.

The purpose of the use and development of KPIs must be disseminated in the organisation, from the lower to the higher- level employees in order to make them understand their own role in helping to achieve the organisation’s goals and the measurement and improvement process.

STEP 4: Establish an agreed process for KPIs development and use.

After having created the team that will be responsible for the KPIs’ development, defined the key goals, which have to be measured and therefore aligned with the KPIs, and communicated and disseminated the purpose of KPIs implementation, an agreed development and use process must be established. The process should include:

• Definition of the team members’ duties

• For each KPI that will be selected by the project team, a set of Input, Calculate, Output and Report Procedures should be reported during the implementation phase as follows:

Input involves all procedures for collecting data and the parameters used for evaluating KPIs.

Calculate involves all mechanism and calculation required for evaluating KPI, from given Input data.

Output involves all processing and presentation of results e.g. graphical forms, data etc.

Report involves all reporting Requirements for monitoring RE procedure improvement. It may consist of a composite report where progress is shown through a number of Output formats

PHASE II - BUILD

STEP 5: Identify critical success factors for the organisation.

Critical success factors are the core competencies or capabilities that must be accomplished for the organisation to achieve its vision. The set of critical success factors related to the process of requirements engineering and the Business Goals already defined, may include the following:


IST-2000-28760 of 14

Business Goals Critical Success Factors

Improve RE Process Maturity § Use of RE Standards

Improve Managerial Satisfaction

§ Improve Traceability of requirements throughout the System Lifecycle

§ Reduce Uncertainty throughout the System Lifecycle

Improve User Satisfaction § Reduce Users’ (internal – external) Complaints

Improve Efficiency of the System Implementation Process

§ Get things right in the first place § Reduce time to market § Implementation within the time table § Keeping within budget

Improve Maintainability of System § Improve extentability § Reduce maintenance cost

Improve Productivity of S/W development

§ Reduce code production related to rework

§ Reduce Effort of Rework

STEP 6: Teams define yardsticks and select KPIs.

Following the definition of Business Goals and critical success factors, a set of yardsticks should be defined. The yardsticks will specify the practical result that each organisation and the project should achieve according to the specified Business Goals. They will also serve in monitoring and evaluating the process towards the achievement of these goals.

A set of Yardsticks for the Business Goals and the Critical Success Factors defined, may be:


IST-2000-28760 of 14

Critical Success Factors Yardsticks

Use of RE Standards § The increased number of RE Standards

Improve Traceability of requirements throughout the System Lifecycle

§ The Existence of Requirements Document

§ The increased number of unique requirements in Requirements Document

§ A high percentage of unique requirements

Reduce Uncertainty throughout the System Lifecycle

§ A low percentage of reworked requirements

Reduce Users’ (internal – external) Complaints

§ The decreased number of modifications coming from RE errors

Get things right in the first place

§ A high percentage of requirements satisfied in the initial design

Reduce time to market § The minimization of time between system conception and delivery

Implementation within the time table

§ The minimization of extra time spent for the implementation

§ The minimization of extra time spent for the RE process

Keeping within budget § A low percentage of deviation from total budget

§ A low percentage of deviation from the RE budget

Improve extentability § The increased number of additional

requirements satisfied / number of additional requirements requested


IST-2000-28760 of 14

Reduce maintenance cost § The lowering in cost of maintenance per 1000

lines of code

Reduce code production related to rework

§ The decreased number of code lines / person day spent on rework

Reduce Effort of Rework § The minimization of the number of person days

needed for rework due to requirements capturing inefficiency

KPIs should have a set of characteristics that will allow them to be effective, namely they should be:

§ Restricted to a few strategically important objectives.

§ Based on strategic goals, making general goals more specific.

§ Consistent, giving a balanced view, relating to financial performance, satisfaction, efficiency and improvement.

§ Uncomplicated and understood by those being measured.

§ Easy to collect data and calculate.

§ Dynamic (reviewed at least annually as a part of business planning, to ensure that they reflect business priorities).

§ Agreed, not imposed by management.

§ Communicated.

KPIs should not:

§ Conflict with other measures without prioritising.

§ Produce misleading information.

§ Be or viewed as trivial.

All the above characteristics should be taken under consideration for the process of the KPIs development.

Suggested KPIs related to the respective Yardsticks, Critical Success Factors and Business Goals are:


IST-2000-28760 of 14

Yardsticks Key Performance Indicators

The increased number of RE Standards Number of RE Standards used

The Existence of Requirements Document Existence of Requirements Document

The increased number of unique requirements in Requirements Document

Number of Unique Requirements

A high percentage of unique requirements % Unique Requirements completed

A low percentage of reworked requirements % Requirements to be reworked

The decreased number of modifications coming from RE errors

Number of modifications coming from RE errors / month

A high percentage of requirements satisfied in the initial design

% Requirements satisfied in the initial design

The minimization of time between system conception and delivery

Time between system conception and delivery

The minimization of extra time spent for the implementation

Number of extra months spent for the implementation

The minimization of extra time spent for the RE process

Number of extra months spent for the RE process

A low percentage of deviation from total budget

% Deviation from total budget

A low percentage of deviation from the RE budget

% Deviation from the RE budget


IST-2000-28760 of 14

The increased number of additional requirements satisfied / number of additional requirements requested

Number of additional requirements satisfied / number of additional requirements requested

The lowering in cost of maintenance per 1000 lines of code

Cost of maintenance per 1000 lines of code

The decreased number of code lines / person day spent on rework

Lines of code/person day spent on rework

The minimization of the number of person days needed for rework due to requirements capturing inefficiency

Number of person days needed for rework due to requirements capturing inefficiency

PHASE III - IMPLEMENT

STEP 7: Develop display, reporting and review frameworks at all levels.

The KPIs should be displayed and reported frequently. The KPIs reporting should show management the trend. Thus, graphics going back are considered a good practice. Such a daily or weekly display will not take very long once the process is embedded.

The measurements collected should be analysed and form the basis for the final report.

STEP 8: Facilitate the use of KPIs to assist performance improvement

Education and training will facilitate the use of KPIs inside the organisation. Employees should be motivated and empowered by giving them controlled authority and responsibility for improvement, in relation to the processes they are responsible for. Managers should be encouraged to delegate responsibility and workers to identify their own measures and look for solutions.

PHASE IV - REVIEW

STEP 9: Refine and modify KPIs to retain their relevance.

The KPIs should be reviewed every time the critical success factors are reviewed. They also KPIs should be checked in terms of their relevance and be adapted to the eventually changing environment, processes and conditions.


IST-2000-28760 of 14

Executives must be actively involved in reviewing the measures and leading the implementation effort. Continuous communication regarding the status and results of the measurement system is important in terms of cont inually improving the system.

2.3. Methodology at a glance

The overall methodology is represented in the figure below:

STEP 9 : Refine and modify KPIs to retain

their relevance

STEP 8 : Facilitate the use of KPIs to assist

performance improvement

STEP 7 : Develop display, reporting and review frameworks at

all levels

STEP 6 : Teams define yardsticks and select

KPIs

STEP 5 : Identify critical success factors

for the organisation


STEP 2: Align KPIs development with key

areas and improvement strategies

in requirements engineering

STEP 3 : Explain the purpose of the use and development of KPIs

STEP 4 : Establish an agreed process for

KPIs development and use

Initiate

Build

Implement

Review


their relevance




all levels


KPIs











their relevance




all levels


KPIs










Initiate

Build

Implement

Review


IST-2000-28760 of 14

3. Appendix A. Indicative Project Team Structure

The project team will be consisted of high- level employees coming from different departments of the organisation, namely:

The Co-ordinator: High ranking official who will undertake the coordination of the team. He has an overall view of the project and he will communicate with the team involved in the project for the definition of KPIs. He will also be responsible for the measurements.

General Manager: He has an overall view of the organisation and he will set the business goals and discuss the priorities and the critical success factors with the rest members of the team, in order to proceed with the definition of the KPIs.

The Production / Technical Manager or high-ranking representative: He coordinates the resources and activities required for the production (in the case of industrial production companies) and the project implementation process (in the case of IT companies). He will be responsible for the data obtained, related the production & the IT implementation process.

The IT Manager or high-ranking representative: He will provide all the necessary knowledge about the technical part that has to be measured. He will also communicate with the software engineers and the developers during the process of the definition, the implementation and the dissemination of KPIs.

The Marketing & Sales Manager or high-ranking representative: He is responsible for the organisation’s promotion, the competition and the improvement of the organisation’s market stake. He will ensure that the decisions made are according to the marketing strategies.

The Financial Manager or high-ranking representative : He will define and monitor the critical financial figures.

Quality Manager or high-ranking representative: He is responsible for the quality assurance of the organisation. He ensures that the procedures followed during the KPIs definition & implementation procedure are according to the organisation’s predefined quality assurance procedures.

All team members will participate in the process of definition, implementation, reporting and reviewing of the KPIs. The indicative team structure is represented in the figure below:


IST-2000-28760 of 14

Co-ordinator

Financial Manager

GeneralManager

ITManager

Marketing & Sales

Manager

QualityManager

Customer

Implement

Report Review

Define

ProductionManager

Co-ordinator

Financial Manager

GeneralManager

ITManager

Marketing & Sales

Manager

QualityManager

Customer

Implement

Report Review

Define

ProductionManager

Impression

Technology

Transcript of Impression