30221653 Project Success Criteria

7/28/2019 30221653 Project Success Criteria

1/22

Project Success Criteria for a South African Power Utility

PUBLISHABLE ARTICLE

Presented to the

Graduate School of Business Leadership

University of South Africa

In partial fulfilment of the

Requirements for the

MASTERS DEGREE IN BUSINESS LEADERSHIP

UNIVERSITY OF SOUTH AFRICA

by

N MOODLEY (30683475)

30 NOVEMBER 2004


2/22

2

1 Introduction

Eskom, a South African power utility, is undergoing a period of rapid growth due to

the inherent increase in electricity demand in the sub-Saharan region. Much of this

growth is directly linked to ensuring that the present plant is reliable and able to

sustain its position for the long term, so as to adequately support the new build

capacity projects. This article focuses on one of the business units of Eskom known

as Peaking. Eskom Peaking assets cover the hydro electric, pumped storage, gas

and wind generating plants of Eskom. Most of the Peaking plant is in the latter stages

of its life cycle and numerous refurbishment projects are planned in the next decade

so as to ensure that peak power capacity is available and reliable.

Problem Statement

Due to the large capital investment in projects over the next five years, management

is faced with the challenge of evaluating the present level of project success and to

develop effective performance measures to ensure the success of projects in the

future. Of utmost importance are the identification of success areas that could be

replicated within the Eskom business and the detection of areas needing

improvement.

Although numerous projects have been completed with some perceived level of

success, management would like to have specific measures in order to quantify such

success. Presently, only a few measures are used to evaluate project success

formally, thus it is paramount to identify what other factors affect the success of

projects.

In this study, the following are a list of questions from management;

What are key performance indicators for ensuring future project success?


3/22

3

Are time, cost and quality the only important project success criteria?

What other factors affect the success of projects?

What are the areas of concern / focus?

Does quality affect project success?

Is adequate time allocated for project execution?

Does meeting the user requirementshave an effect on the success of projects?

Objectives

The aim of this article is to identify archetypal success criteria for engineering and

refurbishment projects so that adequate key performance indicators for project

success are available to management.

The outcome of this study must also highlight the importance or non importance of

relevant secondary project success criteria. The conclusions drawn from this study

should provide a guideline to other power utilities and industries who envisage similar

refurbishment projects or projects of a similar nature.

The final analysis of this study provides an investigation into the management focus

areas so that management can concentrate its efforts on salient issues regarding the

improvement of the project process. The main question posed by management is

whether the traditional time, cost and quality measures are the only criteria to

measure project success. This forms the basis for this study which is composed of

an investigation into the current success criteria identified from a literature review and

interviews. In these interviews, selected criteria are tested on Eskom Peaking

stakeholders to identify the most significant criteria that can be used to evaluate the

success of future projects. The analysis is based on data obtained from


4/22

4

questionnaires and is presented in the form of descriptive and statistical analyses to

identify correlations between the various success criteria and relevance to the

business unit.

Paper PropositionThe following hypothesis will be tested in this article; Time, costand qualityare the

only important success criteria for engineering and refurbishments projects.

2 Theory and Practice of Project Success

Project purpose

It is appropriate to first identify the purpose of a project. Most articles describe a

project as being a unique activity having a clear goal with definite start and finish

dates. The whole reason for creating and implementing a project is to satisfy very

specific objectives of the business. Projects are viewed as being the execution of all

the visions, missions and strategies of an organisation [1]. This concept is further

elucidated by the PMBOKGuide(PMI, 1996), highlighting that all projects should be

supportive of performing the organisations strategic goals. This places emphasis on

the projects meeting and supporting the strategy and long term goals of the business

[2]. In the case of Eskom Peaking, refurbishment projects are created to ensure that

the plant maintains a high level of availability and reliability. This would, in the long

term, ensure that the peak power needs within South Africa and sub-Saharan Africa

are adequately satisfied with the least cost of production. These projects also allow

for future build projects to be given a chance to develop without the strain of being

overly fast tracked. The question is: What are the criteria for ensuring that the project

objectives are met and how are these measured?


5/22

5

Success Criteria

Success criteria are the measures by which success or failure of a project or business

is judged [3]. It is a set of measures that are understood by the relevant stakeholders

and, if performed adequately, would guarantee some level of success. De Wit [3]

further emphasises that project success is measured against the overall objectives of

the project whereas project management success is measured against the traditional

measures of performance against time, costand quality. He further highlights that by

using cost as a measure, one is measuring the progress made and not the level of

success. Baccarini [4] proposes that project success is composed of two distinct

components. One is project management success (time, cost, quality, process) and

the other is product success (effects of the projects final product).

Other authors (deWit [3], Wateridge [5], McCoy [6], Pinto and Slevin [7], Saarinen and

Ballantine [8], Davies [9]) also share the view that time, costand qualityare the basic

measures, but the assessment of project success is not only limited to these criteria.

Some authors are of the opinion that time, costand qualityare measuring the delivery

stage of a project. The post delivery stage, which identifies success criteria set by the

stakeholders [5], also plays an important role.

Stakeholders

The concept of project success has instigated numerous debates on its definition and

scope due to the very nature of its purpose. Does one evaluate the impact of the

project, or the way the project was implemented? Different stakeholders have

different views on what criteria identify the project as being a success [10]. This

would be driven by the unique need that is being satisfied by the project. An opinion

that is clearly identified in the literature review is that project success criteria must be


6/22

6

clearly defined before the project commences so that all stakeholders fully

understand the requirements and can work towards a common goal [5]. According to

Roger Atkinson [11], one reason for projects being perceived as failures is that the

criteria for evaluation have changed, yet the same old criteria are still being used to

assess project success.

Different stakeholders tend to place focus on different project criteria. Struckenbruck

[12] has suggested that the four most important stakeholders to decide on the

success criteria are the project manager, senior management, customer-client and

the project team members. The project manager would be content if the project

duration and budget are met with reasonable quality. The customer of the project is

satisfied if the user requirements are met with no, or insignificant, post-project

defects. Senior management focuses on how the project has contributed towards

achieving the strategy of the business and its alignment to achieving the company

goals. Van Aken [13] defines project success as the satisfaction of all stakeholders.

Research on project success suggests that it is extremely difficult to identify generic

project success criteria that can be used for the evaluation of all projects. Even

though some projects can be grouped into common types, the stakeholders have the

final say on the performance of the project. Each stakeholder brings his/her unique

opinions and perceptions of project success. The culture of the organisation also

plays a pivotal role towards defining inherent project success perceptions [14].

Alternative Criteria

Studies have shown that there are identifiable factors that are critical to ensure the

success of projects [15]. These are: communicationthroughout the project, clear

objectives and scope, breaking the project into bite sized chunks and using project


7/22

7

plans as working documents. Morris and Huge [16] have highlighted that the success

of a project depends on the following factors: a realistic goal, competition, client

satisfaction, a definite goal, profitability, third parties, market availability, the

implementation process and the perceived value of the project.

Shenhar, Levy and Dvir [17] have identified four independent dimensions of project

success. These are: project efficiency, impact on customer, business success and

preparing for the future. Project efficiency covers both the timeand costaspect of

the traditional measures. Impact on the customer encompasses the meeting of the

technical and operational specifications of the project, fulfilling the customers needs,

actual use by the customer, solving of the customers major problems and, finally, the

overall aspect of customer satisfaction. The business success criteria for internal

projects include measures such as yields, cycle times, processing steps and quality.

The fourth dimension, which is preparing for the future, includes a measure to ensure

that the internal project contributes effectively towards developing new technology,

skills or competencies.

From studies carried out by Wateridge [5], it was found that for all stakeholders

involved, the following were the success criteria in order of importance: meet user

requirements, happy users, achieves purpose, meets budget and meets time.

3 Research Methodology

This study employs a normative investigative style by sending a questionnaire with

statements ranked on Likert scales. The criteria listed in the questionnaire are those

that were identified and deemed appropriate from the literature research and other

criteria suggested by the stakeholders in preliminary interviews. The questionnaire


8/22

8

also allowed for an open ended question which required the respondent to list any

other criteria that they thought relevant for the measure of project success. The

questionnaire was divided into two sections and designed to allow comparisons to be

drawn between the relevant project stakeholders. Questionnaires were sent and

received via e-mail. This was the most convenient and quickest means of response

due to Eskom Peaking having six power stations that are geographically dispersed

throughout South Africa. The survey concentrated on respondents in the business

that were directly involved in, or affected by, projects implemented. The following

stakeholders were identified: senior/line management, production/client, commercial,

finance, project managers and system/project engineers. Questionnaires were

initially tested on five respondents to identify whether they were easy to understand

and complete. Eskom Peaking consists of 241 employees, of which 62 employees

were directly affected by the projects. Questionnaires were sent to those 62

employees and a response was received from 37 of them, with a response rate of

59.67%. This was considered adequate for this study.

The following criteria were identified as the most relevant for Eskom Peaking by

means of initial interviews with key stakeholders and by the literature review. These

were used in the questionnaire:

o Time- Completing the project in time

o Cost- Completing the project within budget

o Quality - In the context of Eskom Peaking, quality is defined as meeting the

technical specification, no defects after one month of installation and having all

documentation in place for implementation and at close out of the project.

o User requirements - Meeting the customers specifications and requirements

for the project.


9/22

9

o Customer satisfaction Satisfying the needs of the customer. This is more

than just satisfying the user requirements.

o Satisfy business goals- Does the project satisfy the goals and strategy of the

business?

o Communication This criterion identifies if there was adequate communication

throughout the project. It was deemed important by some clients who needed

to be kept informed of all aspects of progress on the projects implemented on

their plant.

o Impact on plant This criterion identified the impact that the project would

have on the plant in terms of the unit performance, availability and reliability.

o BEE This criteria was included since it was a performance criteria prescribed

by the Eskom corporate office to improve the level of support for black

businesses in the market. BEE stands for Black Economic Empowerment

companies.

o Contractor management the success of the project was also linked to how

the contractor was managed during the implementation phase of the project.

This could have an impact on resources, planning and contractual issues

which was considered important enough to include as a success criterion.

4 Research Observations

Figure 1 summarises the number and percentage make-up of the respondents who

participated in the questionnaire survey. An adequate number of responses were

received from the relevant stakeholders making the survey representative of the

business.


10/22

10

Figure 1: Respondent Make-up

Respondents No. %

Senior/Line manager 7 19%

System/project engineer 13 35%

Production/Client 8 22%

Project manager 4 11%

Finance 2 5%

Commercial 3 8%Total 37 100%

Figure 2 summaries the responses on the Likert scale for all respondents where

averages above 4.00 are considered significant. The results indicate some level of

concurrence on the relevance of the criteria chosen from the literature review and

initial interviews, since all success criteria except BEE and contractor management

were rated above 4.00. The BEE rating was consistently rated low by all

stakeholders. The highest rating was quality, which was closely followed by impact

on plant, satisfy business goalsand time. Customer satisfaction, user requirements,

communicationand costwere the next set of criteria rated above 4.00.

Figure 2: Likert averages for all responses

4.34

4.03

4.63

4.20

4.26

4.34

4.06

4.50

3.06

3.91

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

Time

Cost

Quality

User Requirements

Customer Satisfaction

Satisfy business goals

Communicaion

Impact on Plant

BEE

Contractor Management

Comparisons made between the main stakeholders revealed important focus areas.

Figures 3-5 indicates the Likert averages for the production/client, project managers


11/22

11

and senior/line management respectively. The client places significant focus on

satisfy business goals, customer satisfaction and impact on plant which were rated

significantly higher than timeand quality. Other positive criteria were communication,

contractor managementand cost. The lowest rating was BEE, which was consistent

with the overall results.

Figure 3: Likert averages - Production/ client

4.38

4.00

4.50

4.38

4.63

4.75

4.13

4.63

3.00

4.13

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

Time

Cost

Quality

User Requirements



Communicaion

Impact on Plant

BEE


The project managers displayed different opinions and placed high focus on quality.

The other significant criteria which were rated the same were time, cost, user

requirements, and impact on plant. These were consistent with the notion that project

managers concentrate on the immediate outputs of the project which measures direct

progress instead of the long term goals. They awarded a low score for customer

satisfaction, satisfy business goalsand communication. Contractor managementand

BEE were both rated below 4.00. It was considered unusual that the project

managers did not focus highly on contractor managementas it was one of their key

functions.


12/22

12

Figure 4: Likert average - Project manager

4.50

4.50

5.00

4.50

4.25

4.00

4.00

4.50

3.00

3.75

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

Time

Cost

Quality

User Requirements



Communicaion

Impact on Plant

BEE


Senior/Line management rate satisfy business goalsand qualityas high focus areas;

with time, customer satisfaction, and impact on plantrated equal third highest. User

requirements, communication, contractor managementand BEEwere all rated below

4.00 indicating that management does not place much focus on these measures.

Figure 5: Likert average - Senior/line management

4.29

4.00

4.57

3.86

4.29

4.57

3.71

4.29

3.29

3.57

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

Time

Cost

Quality

User Requirements



Communicaion

Impact on Plant

BEE


Figure 6 depicts the response from stakeholders on their perception of time, costand

quality being the only success criteria for projects. The results indicate that 62% of


13/22

13

the respondents think that these are not the only criteria for evaluating project

success. An interesting summary is obtained when these results are broken down

according to the individual stakeholders. It is clearly demonstrated that senior/line

management and system/project engineers are of the opinion that time; cost and

quality are definitely not the only criteria whereas the project manager, finance and

commercial departments are of the contrary opinion. Production/client is indifferent to

these criteria and considers the other highlighted criteria to be just as important as

time, costand quality.

Figure 6: % Response - "Are Time, Cost, Quality the only success criteria?"

Respondents Yes NoSenior/Line manager 0% 100%

System/project engineer 23% 77%

Production/Client 50% 50%

Project manager 75% 25%

Finance 100% 0%

Commercial 67% 33%

Overall 38% 62%

An interesting point is noted from Figure 7. Respondents who answered Yes to the

question Are time, costand qualitythe only success criteria?, scored higher for all

criteria except for customer satisfaction and satisfy business goals. This gives the

impression that respondents who answered No were more interested with the long

term goals of the project and how it satisfied the needs of the stakeholders.


14/22

14

Figure 7: Yes/No Likert average scores

4.43

4.79

4.50

4.21

4.21

4.07

4.57

3.14

3.93

4.30

3.87

4.48

4.00

4.30

4.43

4.04

4.22

3.00

3.87

4.29

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

Time

Cost

Quality

User Requirements



Communicaion

Impact on Plant

BEE


Average Yes Average No

A Pearson moment correlation was performed on the project success criteria to

identify possible correlations and relationships between the criteria. The critical

significance level is taken as


15/22

15

Cost is positively correlated with quality and BEE. Quality demonstrates positive

correlation with satisfy business goals, communication and impact on plant. User

requirementcorrelates positively with customer satisfaction, impact on plantand BEE.

Communication correlates positively with impact on plant, BEE and contractor

management, with the highest correlation being with contractor management. BEE

and contractor managementcorrelates positively with impact on plant, with BEEand

contractor managementalso being positively correlated.

Figure 9 summarises the response from stakeholders when asked if they could

highlight alternate criteria not mentioned in the questionnaire. These criteria were

grouped into the relevant categories so as to identify common dimensions of criteria.

Knowledge transfer was highlighted by both project engineers and project managers.

When questioned on this response, stakeholders highlighted that there was a need

for personal development and growth of project team members.

Figure 9: Success Criteria Highlighted by Respondents

Other Criteria Highlighted Frequency

Knowledge Transfer 4Value for Money 4

Project Management Style 3

Working relationship 2

Effective Project Planning 2

Long Term Impact 2

Value for money is another criterion that respondents have highlighted as being a

meaningful indicator of project success. This is closely linked to achieving the

business goals at the least cost with the most positive impact on the plant. Project

management style is a further aspect that has been highlighted for consideration as a

project success criterion. Respondents further highlighted that this factor could have

a direct impact on communication, contractor managementand customer satisfaction.


16/22

16

Other criteria that were mentioned were working relationships, effective project

planning and long term impact of the project.

5 Discussion

From the results obtained in the questionnaire survey, it is clear that time, costand

qualityare not the only criteria for measuring project success. This is consistent with

most studies carried out on project success [3, 5-8]. The questionnaire response

from the different stakeholders re-enforces the choice of criteria selected, as the

averages for all criteria were above the rating of 4.00 for all respondents, except BEE

and contractor management. The BEE criterion is viewed as a short term

performance criterion, which is more of a corporate indicator than one of strategic

business requirements. The low rating was consistent among all respondents.

Although this may be an important business target it has not been a major focus area

on projects, as this was traded off with the other important criteria.

This study also highlights that different stakeholders have different criteria to evaluate

project success. The project manager is concerned more about the short term

requirements and about delivering the project within its constraints. Management is

concerned more about the long term and strategic requirements. The customer

focuses on the impact of the project and how it satisfies his/her needs. Measurement

of their performance can then be based on these criteria. In this way the project team

will be able to adequately focus on the salient measures to ensure project success.

The criteria highlighted in this study would then ensure that both the needs of the

business and customers are met.


17/22

17

The five most important project success criteria highlighted for projects within Eskom

Peaking are: quality, impact on plant, satisfies business goals, time and customer

satisfaction. Although cost is not one of the five most important highlighted, it would

form part of the evaluation criteria so that sufficient emphasis is placed on effectively

managing the cost of a project. This is due to the fact that Eskom, like most power

utilities, strives to generate the lowest cost electricity which impacts on the strategic

requirements of the business, industry and the Southern African region. It is felt that

management would gain enormously by using the identified success criteria to

understand and formulate how future projects contribute towards creating value for

the business and improve the processes for managing future projects.

Qualitywas rated consistently higher by all stakeholders. This is consistent with the

culture portrayed within the Eskom Peaking business unit. The division recently

undertook the ISO9001 rating and the emphasis placed on quality over the past few

years has been significant. Quality has a direct impact on the plant, which could

affect the availability and reliability of the power stations adversely if projects are not

implemented properly or have defects which could result in plant failures. Qualityis

also positively correlated with satisfy business goals, communicationand impact on

plant. This is consistent with the belief that the better the quality, the more the plant is

positively affected and in this way satisfies the goals of the business. This

relationship also emphasises that the better the communication between all project

stakeholders, the higher the level of quality of the project. It is felt that quality is a

significant success criteria and management must ensure that the level of qualityof

projects are always maintained at a high level.


18/22

18

The criterion impact on plant plays a major role for the reliable supply of electricity

especially during emergency and peak power loading on the power network. These

are critical demand periods when Peaking plant would be called upon and projects

implemented must ensure that the plant is impacted on positively with respect to

performance and availability. This criterion is deemed relevant and consistent with

the stakeholder ratings. Satisfying the business goals is quite important especially if

it calls for long term sustainability of the plant and a strategy that ensures that Eskom

as a whole has enough capacity to satisfy the rapidly growing demand of electricity.

Projects must be implemented in a manner that effectively supports this need.

Time, which is one of the three traditional measures, is considered relevant especially

when projects are on the critical path of a plant outage (down time). The sooner the

projects are implemented, the sooner the plant is available for production. Time is

directly linked to the plant availability and is the key component and emphasis within

the power utility business, especially with the current growth in electricity demand

within the sub-Saharan region. This ensures that plant capacity is always available

due to power constraints on the power network.

The positive correlation of time to user requirements and customer satisfaction

confirms that projects that are implemented within the allocated time satisfy the

customer as they meet their target of managing the allocated time in which the plant

has to be off-line, which is also one of productions/clients key user requirements. The

other aspect of this correlation is that the more time spent on the plant, the greater

the effect or impact there is on the plant. This could however have a negative impact,

thus the time spent on the plant needs to be effectively controlled. Customer

satisfactionhas always been a success criterion within the Eskom Peaking business,


19/22

19

especially due to the project team being based centrally and offering a service to six

power stations. Internally, Production is the key customer. They are also directly

accountable for the plant availability and reliability, and the users of the plant and

project outputs. These were consistent with the performance targets that were placed

on senior/line management. They were expected to ensure high availability and

reliability of the plant, that the customers electricity requirements were adequately

met with the long term sustainability of the plant and equipment (high quality).

Knowledge transfer was highlighted as a focus area for management owing to most

of the existing project managers and engineers having inadequate experience in

project and contract management. This was due to their being exposed to the project

environment for less than two years.

6 Conclusion

The main conclusion from the results of this study is that there are many more

success criteria that are relevant for evaluating the success of projects and one is not

only limited to the conventional time, cost and quality. The five relevant project

success criteria highlighted in this study are quality, impact on plant, satisfies

business goals, time and customer satisfaction. Stakeholders play a key role in

assessing the level of project success and it is quintessential to first define and agree

on the success criteria at the initiation stage of the project so that all stakeholders are

aware of the focus areas, the purpose and benefits of the proposed project.

Recommendations

The following recommendations are made to Eskom Peaking management:

o The relevant stakeholders for each project must be clearly identified at the start of

the project.


20/22

20

o Project success criteria must be discussed and agreed upon with the relevant

stakeholders at the start of the project.

o The most relevant project success criteria for Peaking projects are quality, impact

on plant, satisfy business goals, time and customer satisfaction. Management

should use these measures to glean the project success and the level of

alignment of projects to the business strategy.

o The cost of the project must also form part of the measure to ensure optimum

cost, and that the return on the project is justified.

o Adequate project auditing must be carried out at the completion of the project, and

the success criteria must play a key role in the evaluation of what was initially

agreed with all stakeholders.


21/22

21

7 References

1. Maylor H. 2003. Project Management, Prentice Hall, 3rd Edition.

2. Meredith JR. and Mantel, S. J., 2000. Project Management A managerial

approach, John Wiley & Sons, 4th Edition.

3. de Wit A. Measurement of project management success. International Journal of

Project Management, 1988, 6(3), 164-170.

4. Baccarini D. The logical framework method for defining project success. Project

Management Journal, 1999, 30(4), 25-32.

5. Wateridge J. How can IS/IT projects be measured for success? International

Journal of Project Management, 1998, 16(1), 59-63.

6. McCoy FA. Measuring success: Establishing and maintaining a baseline, Project

Management Institute Seminar/Symposium Montreal Canada, Sep. 1987, 47-52.

7. Pinto JK and Slevin DP. Critical success factors across the project lifecycle.

Project Management Journal, 1988, XIX, 67-75.

8. Saarinen T. Systems development methodology and project success, Information

and Management, 1990, 19, 183-193.

9. Davies TC. The real success factors on projects. International Journal of Project

Management, 1999, 17(6), 337-342.

10. Lim CS and Mohamed MZ. Criteria of project success: an exploratory re-

examination. International Journal of Project Management, 1999, 17(4), 243-248.

11. Atkinson R. Project management: cost, time and quality, two best guesses and a

phenomenon, its time to accept other success criteria. International Journal of

Project Management, 1999, 17(6), 337-342.

12. Struckenbruck L. Who determines project success. Project Management Institute

Seminar/Symposium Montreal Canada, September 1987, 13-21.


22/22

13. Van Aken T De Weg naar project success: Eerder via werkstijl dan instrumenten,

De Tijdstroom, 1996.

14. Gray RJ. Organisational climate and project success. International Journal of

Project Management, 2001, 19, 103-109.

15. Clarke A. The key success factors in project management. Proceedings of a

Teaching Company Seminar, London, December 1995.

16. Morris PWG and Hugh GH. Preconditions of success and failure in major projects,

Templeton College, the Oxford Centre for Management Studies, Kinnington

Oxford, Technical paper No. 3, September 1986.

17. Shenhar AJ, Levy O and Dvir D. Mapping the dimensions of project success.

Project Management Journal, 1997, 28(2), 5-13.

30221653 Project Success Criteria

Documents

Transcript of 30221653 Project Success Criteria