A Study of Building an Information Systems Project Risk Checklist...
Transcript of A Study of Building an Information Systems Project Risk Checklist...
A Study of Building an Information Systems Project Risk Checklist by Analyzing Critical
Factors of Information Systems Failures
A study submitted in partial fulfillment of the requirements for the degree of Master of Science in Information Management
at
THE UNIVERSITY OF SHEFFIELD
By
Lihong Zhou
September 2006
- 1 -
Abstract
Information systems within organizations were rapidly developed and widely used in
diverse industries in recent decades. However, currently, the astonishing rate of
information system failures becomes a controversial issue and a pertinent research
topic. Although a large amount of studies have contributed to this topic, there is also
a large number of information systems are not as successful as expected. The main
purpose of this dissertation is to create an Information Systems Project Risk
Checklist as a convenient elementary risk identification device for information
systems developers in avoiding and mitigating potential project risks in order to keep
information systems project away from failure.
This dissertation research is conducted by using an inductive research methodology
with secondary data source and qualitative data analysis approach. Both conceptual
and empirical study are carried out in this study by literature review and case study.
After Introduction and Research Methodology chapters, the extensive literature
review, on Information System Implementation, Nature of Failure, Project
Management and Risk Management, buildup a research theory background. The
initial version of Information Systems Project Risk Checklist is generated based on
this theory foundation.
The formal version of Information Systems Project Risk Checklist is presented after
2 case studies (Integrated National Crime Information System Project (INCIS) and
London Ambulance Service (LAS)). The formal version risk checklist is significant
revised from the initial version by applying the risk checklist into real life
environments.
- 2 -
Acknowledgement
I would like to give my extreme appreciation to my supervisor Ms Ana Cristina
Vasconcelos for her encouragement, guidance and assistance throughout the entire
project. Without her support the dissertation could not have been completed.
- 3 -
Contents
Abstract....................................................................................................................... 1 Acknowledgement ...................................................................................................... 2 Contents ...................................................................................................................... 3 1 Introduction ............................................................................................................. 6
1.1 Current Study ....................................................................................................................... 7 1.2 Research Motivation and Objectives.................................................................................. 10 1.3 What is Checklist ?............................................................................................................. 11 1.4 Why Study Failures? .......................................................................................................... 11
2 Research Methodology.......................................................................................... 13 2.1 Literature Review............................................................................................................... 14 2.2 Case Studies ....................................................................................................................... 14 2.3 Research Approach............................................................................................................. 16 2.4 Data Collection and Analysis ............................................................................................. 18 2.5 Dissertation Outline and Structure Map............................................................................. 20
3 Information Systems Implementation................................................................. 22 3.1 Definition of Information System ...................................................................................... 22 3.2 Nature of Information Systems .......................................................................................... 23 3.3 Importance of Information Systems ................................................................................... 26 3.4 Information Systems in Organizations ............................................................................... 27 3.5 Information Systems Development Methodologies ........................................................... 30
4 Nature of Failure ................................................................................................... 33 4.1 Definition of Failure........................................................................................................... 33 4.2 Types of Failure.................................................................................................................. 34 4.3 Information Systems Failures............................................................................................. 34
5 Project Management............................................................................................. 37 5.1 Managing Information Systems Projects ........................................................................... 38 5.2 System planning and estimating......................................................................................... 40 5.3 System Design, Development and Implementation ........................................................... 43 5.4 System Implementation, Testing and Maintenance............................................................ 45
6 Risk Management ................................................................................................. 48 6.1 Risk Management Process ................................................................................................. 50 6.2 Risk Identification.............................................................................................................. 51 6.3 Risk Assessment and Management .................................................................................... 52
7 Information Systems Project Risk Checklist ...................................................... 55 7.1 Literature Review Based Checklist Building ..................................................................... 55 7.2 Pre-Project.......................................................................................................................... 57 7.3 Customer ............................................................................................................................ 58 7.4 Project Management........................................................................................................... 59 7.5 Project Technical Aspect .................................................................................................... 60 7.6 System Development ......................................................................................................... 61
- 4 -
8 Case Study 1: Integrated National Crime Information System (INCIS)......... 63 8.1 Case Background Introduction........................................................................................... 63 8.2 Problem Domains Description ........................................................................................... 64
8.2.1 Pre-project ................................................................................................................... 64 8.2.2 Customer ..................................................................................................................... 68 8.2.3 Project Management.................................................................................................... 70 8.2.4 Project Technical Aspect ............................................................................................. 72 8.2.5 Project Development ................................................................................................... 75
8.3 Conclusion........................................................................................................................... 76 9 Case Study 2: London Ambulance Service ......................................................... 77
9.1 Case Background Introduction........................................................................................... 77 9.2 Problem Domains Description ........................................................................................... 78
9.2.1 Pre-Project................................................................................................................... 78 9.2.2 Customer ..................................................................................................................... 79 9.2.3 Project Management.................................................................................................... 82 9.2.4 Project Technical Aspect ............................................................................................. 85 9.2.5 Project Development ................................................................................................... 87
9.3 Conclusion.......................................................................................................................... 89 10 Formal Information Systems Project Risk Checklist ...................................... 92
10.1 Pre-Project........................................................................................................................ 92 10.2 Customer .......................................................................................................................... 94 10.3 Project Management......................................................................................................... 96 10.4 Project Technical Aspect .................................................................................................. 98 10.5 Project Development ........................................................................................................ 99
11 Conclusion and Future Research..................................................................... 101 11.1 Research Objectives ....................................................................................................... 101 11.2 Research Processes......................................................................................................... 102 11.3 Research Methodology................................................................................................... 103 11.4 Research Result .............................................................................................................. 104 11.5 Research Contribution and Future Research .................................................................. 105
Reference................................................................................................................. 108 Appendix I .............................................................................................................. 117 Appendix II ............................................................................................................. 121
- 5 -
List of Figures
Figure 1: Sauer’s Triangle of Dependences ................................................................. 9
Figure 2: Research Processes ..................................................................................... 13
Figure 3: Deductive Reasoning.................................................................................. 16
Figure 4: Inductive Reasoning ................................................................................... 17
Figure 5: Quantitative VS Qualitative From (Saunders, 2000:380) .......................... 19
Figure 6:Dissertation Structure Map.......................................................................... 21
Figure 7: Information Systems (Developed from Robson, 1997).............................. 22
Figure 8: A model of system (Curtis and Cobham, 1995:15) ................................... 23
Figure 9: General of Information System (Robson, 1997:84) ................................... 24
Figure 10: Model of Computerized Information System (O’Brien, 2006) ................ 25
Figure 11: Duality of Structure (Beynon-Davies, 2002: 220).................................... 27
Figure 12: The elements and context of a computer-based information system........ 28
Figure 13: Failure Types (Fortune &Peters, 1995: 23) .............................................. 34
Figure 14: Determinants of Information systems Success/failure ............................. 35
Figure 15: An example of Gantt chart from University of Sheffield (2000),Page 2................ 42
Figure 16: The Business Excellence Model of EFQM (Yeates and Cadle, 1996:171) ............ 43
Figure 17: The Risk Management Process (Yeates and Cadle, 1996: 190) ............... 50
Figure 18: Risk assessment process ........................................................................... 50
Figure 19: Risk Impact Scale (Developed from Yeates & Cadle (1996)).................. 53
Figure 20: Risk Likelihood Scale (Developed from Yeates & Cadle (1996)) ........... 53
Figure 21: Diagram of revisions on Information Systems Project Risk Checklist .... 56
Figure 22: Relations of 5 components ....................................................................... 56
Figure 23: Compare INCIS and LAS......................................................................... 90
Figure 24: Diagram of revisions on Information Systems Project Risk Checklist .. 101
Figure 25: Dissertation Structure Map..................................................................... 102
Introduction Lihong Zhou
- 6 -
Chapter 1
Introduction
Ever since the emerging of computer technology and Internet, the world has been
changed substantially. As the continuing evolvement of Internet and the combination
of computer technology and computer network, computers are playing an extremely
important role in people’s everyday work and life. An estimation pointed out by
GobalReach research agency, states that there were upwards of 700 million Internet
users during the year 2004-2005 (Hurst, 2006). Currently, computers in business
world are no longer just information storage and calculation tools, but also powerful
strategic tools, for instance, in the fields of communication, decision support, and
high level management tools. Galliers (1992) indicates computer based information
systems can support all organizations function effectively. Flowers (1996) also claim
that information systems are at the heart of all modern organizations. As the vital
importance of an effective information system within organizations, organizations
pay great amount of money and efforts on the implementation of information
systems.
Since 1960s, a number of information systems have been developed in both private
sectors and public organizations. In the most recent decades, the number has been
dramatically increased. However, the high rate of information system failure is a
pertinent topic. Recent decades of studies on failures of information system
development haven’t substantially decreased the failure rate. Connolly, Begg, and
Strachan (1996) indicates that only in 1996, about 30%-40% of IS development
projects failed or totally abandoned. Figures from survey conducted by Robbins
Gioia in the year of 2001, which covers industries of government, information
technology, communications, financial, utilities and healthcare, pointed out that 51%
ERP systems were viewed as unsuccessful; 46% ERP system were not working
Introduction Lihong Zhou
- 7 -
properly as they designed to be (Aiken, 2002). Forester and Morrison (1994) shows,
a study from U.S. government’s General Accounting Office (GAO), 47% of projects
were delivered but not used, 29% were paid for but not delivered, and 19% were
abandoned or reworked. Martin Cobb pointed out a paradox of “we know why
projects fail, we know how to prevent their failure, so why do they still fail?” (The
Standish Group, Unfinished Voyages, 2006).
A possible reason of failures on information system is unsuccessfully identify and
resolve underlying risks within the system development. A formal and complete risk
management plan and a delicate project management can significantly save the
information system project from the failure. Yeates and Cadle (1996) claims that
before actual system developing, if it is not possible to eliminate potential risks
altogether, it is necessary to recognize the existing risks and make out a plan of
method of dealing with these risks when they actually take place.
1.1 Current Study
Possible risks and causations of information system failure are listed by various
articles. Sumner (1999) lists 12 distinct kinds of failures of information system
implementation: “resource failure; requirement failure; goal failure; technique failure;
user contact failure; organizational failure; technology failure; size failure; people
management failure; methodology failure; planing and contact failure; personality
failure”.
Kasser (1998) exhibits 7 most priorities out of 33 items of risk indicators during
system developing and implementation: poor requirements; poor communications
with customer; lack of, or, poor plans; lack of process and standards; lack of
management support; failure to validate original specification and requirement;
political consideration outweigh technical factors.
Introduction Lihong Zhou
- 8 -
Drori (1997) not only divides an information system implementing project into 5
stages, but also points out negative factors for each stage. Stages Negative factors
System analyzing and requirement defining
1, incomplete comprehension of the system and the current situation of the organization; 2, only focusing on the drawbacks of current system may cause ignoring and excluding new features of the new system; 3, Partially understand the user requirements; 4, User requirements are formed only by management level withough the final system users;
System designing 1, inadequate initial general planing before detailed and complex designing; 2, too accrate designing causes the system extremely heavy and unhealthy; 3, without a prototype agreed by the customer; 4, poor communication and understanding among the system analyst, designer and the programmer;
System developing and acceptance testing
1, attempting produce a complete and perfect system in one stage; 2, initiate system programming when the system designing is not completed; 3, initiate system programming when the data structure of the system is not accurately defined; 4, accept testing, which is finished only finished by the programming team; 5, accept test result only by the management level without considering the final user of the customer organization; 6, programming without a preselected and unified methodology; 7, programming withou necessary documentations; 8, lack of considering reuseable components from the current system; 9, programmer-oriented system designing instead of user-oriented;
System training 1, swich to the new system all around the organization in the mean time; 2, one time training before system cut over; 3, training in a few condensed days; 4, initiate training without complete testing, including the treatment of occationally exceptional problems; 5, lack of user support in the working environment;
System maintainning 1, lack of a thorough maintenance plan; 2, assign unqualified staff for maintenance;
Introduction Lihong Zhou
- 9 -
Nonetheless, Drori’s study only focus on identifying risks in 5 main phases of system
development. These risks are vitally critical, but they are inadequate for a complete
information project.
Sauer (1993) developed an information system failure model, which is named
Triangle of Dependencies. The model is consisted of 3 roles in information system
project: system, supporters and project organization.
(Sauer, 1993: 29)
Figure 1: Sauer’s Triangle of Dependences
Although Sauer’s model makes a positive contribution to comprehend the
information failure, it is not sufficient in identifying a specific risk or a critical factor.
These studies outstandingly exhibit potential risks during the developing of an
information project. However, their surveys are failrly incomplete and partial since
they mainly focus on most significant risks of information systems projects failures.
However, an information system project failure is caused by a series of inseparable
risks but not caused by a single significant risk. It is believable that a comprehensive
risk evaluation device such as a risk checklist can directly shows possible risks in
information systems project, is more practical and convinence for project developers
in planning, designing and implementing information systems.
Introduction Lihong Zhou
- 10 -
1.2 Research Motivation and Objectives
The purpose of the thesis is to build up a complete risk checklist, which can identify
most common risks underlying in information system project, by taking literature
review and assessing 2 case studies of information systems failure in large public
organizations (Integrated National Crime Information System Project and London
Ambulance), discuss the differentiations of these two cases; refine and collect crucial
factors possibly induced the system failure;
The objectives of the dissertation can be highlighted as:
1. Research and evaluate the relevant current literature to establish the nature of
information system, system failure, project management and risk management.
2. Identify critical factors of information system failure from previous publications.
3. Analysis 2 case studies of Integrated National Crime Information System Project
and London Ambulance and identify underlying risks.
4. Formulate an Information Systems Project Risk Checklist which could be an
elementary tool for information systems developers in avoiding potential risks.
5. Suggest future enhancement and development based on this research.
Introduction Lihong Zhou
- 11 -
1.3 What is Checklist ?
As the main purpose of this thesis is to generate an information system project risk
checklist by research on areas related to information systems implementation and
failures, it is necessary to clarify what is a checklist.
According to Stufflebeam (2000), checklists are valuable evaluation devices when
carefully developed, validated, and applied. A sound evaluation checklist clarifies the
criteria that at least should be considered when evaluating something in particular
area; aids the evaluator not to forget important criteria; and enhance the assessment is
objectively, credibility and guiding its operation; and assessing its out comes. In the
evaluation vernacular, checklists are useful for both formative and summative
evaluations.
1.4 Why Study Failures?
Failures are the pillars of successes. The intention of studying failure is to avoid the
certain failure causations and highly improve the success rate in the future by
reassessing the strategic plan and project processes from the past.
Fortune & Peters (1995) illustrates the importance of learning from failures by
exhibiting a statement of Ackoff (1994):
“When one does something right, one only confirms what is already known: how to
do it. A mistake is an indicator of a gap in one’s knowledge. Learning takes place
when a mistake is identified, its producers are identified and it is corrected”
(Fortune & Peters, 1995: 5)
Introduction Lihong Zhou
- 12 -
Lyytinen and Robey (1999) stress on the importance of learning from failures of
information systems development. They mentioned that in order to avoid similar
failure causations in the upcoming project, it is greatly important to for information
systems developers to learn reasons of failures both from internal and external cases
and to effect changes in their own actions.
To conclude, the study of information system failures from the past provides a rich
source of lessons which are worth of investigating, learning and understanding.
Research Methodology Lihong Zhou
- 13 -
Chapter 2
Research Methodology
This research project employs an inductive qualitative research methodology. Mainly,
there are 2 approaches for this study, literature review and case studies.
Figure 2: Research Processes
This dissertation study starts from profound literature review, in order to construct a
research theory background for further empirical studies. More importantly, the
initial version of information system project risk checklist is built base on the
literature review. The purpose of these 2 case studies is to implement the initial risk
checklist to real life cases, and attempt to identify potential risks by using the risk
checklist. The main deliverable, a revised version of information system project risk
checklist, is presented after cases analyzing. The original risk checklist is
significantly revised by risks assessments in the 2 cases.
This chapter aims to exclusively explain the research instruments, research sources
and the overall research plan. At the beginning, 2.1 and 2.2, methodologies of
literature review and case study are discusses. Section 2.3 presents a research
Research Methodology Lihong Zhou
- 14 -
methodology framework, in which substantially discusses inductive and deductive
methodologies. The Secondary data sources are also explained in this section with
qualitative and quantitative data analysis approaches section 2.4. Finally, section 2.5
introduces the structure of the dissertation and a dissertation structure map is
illustrated.
2.1 Literature Review
The literature review is a vital part of this dissertation as previously introduced.
Literature review not just reviews current literatures, but also creates a fundamental
knowledge platform for further cases analysis and research result support. Literature
review goes through basic concepts of information system implementation, nature of
failure, project management and risk management. An information system project
risk checklist is created at the end of literature review.
Based on Yin (1994) “Budding investigators think that the purpose of a literature
review is to determine the answer about what is known on a topic; in contrast,
experienced investigators review previous research to develop sharper and most
insightful questions about the topic” (Yin, 1994 quoted in Liu, 2002).
2.2 Case Studies
Case studies of Integrated National Crime Information System Project (INCIS) and
London Ambulance Service (LAS) are selected for this research.
The case of INCIS was took place in New Zealand. It concerned by public because
of it’s over ambitious strategic plan, high risk project profile and high project cost
(the estimated total cost of the project is NZ$110,000,000). According to the final
Research Methodology Lihong Zhou
- 15 -
report, Small (2000), of the project, although the INCIS project was considered as a
high risk project, the project has the capability to use conventional technology with
enough time, appropriate governance and management, adequate resource, skill and
experience. However, the project was almost certainly failed. It achieved some of its
objectives but failed many of its main objectives.
London Ambulance is a well-known case of IS system failure. Not just the fact of
high estimated financial cost of £1.1–£1.5 million, but more importantly 20 people
may have died as the reason of system failure. Beynon-Davis (1999) claims that this
case is the most frequently quoted UK examples of information system failure.
These two cases are both well-known information system disasters with large scale
of project scopes, high project costs and high technical complexity. And they can
entirely represent large percent of common risks of information system implementing.
Results of these 2 cases are dramatic, but on the other hand, which can increase
public interests. Both of these 2 case studies are from Government Public Sectors.
Although there is a high profile nature of information system failure in private
sectors and companies, there has been very little academic research focused on them,
because of private companies unwilling to disclose details about their failures, as this
kind of information may detrimental to companies’ reputation. However, these kinds
of behaviors increase the possibility of repeat the mistakes for other private sectors.
Galliers (1992) indicates that the methodology of case study is the most popular
strategy in information system failure studies. This book defines 2 groups of
information systems research strategies. The first group is empirical strategy which
includes case studies, field studies, field tests, laboratory studies. Another group is
non-empirical, which is including conceptual, tutorial, review, and others. From the
research results of citation frequency study and published information system articles,
the case study methodology is the most common approach. Galliers (1992) outlines
Research Methodology Lihong Zhou
- 16 -
the both of the bright side and dark side of this strategy. As the strongpoint, case
study captures greater details from the environment of reality. On the other hand,
case study strict to a single project or organization; hard to generalize identified
problems; lack of control of variables; the case can be interpreted distinctly by
different researchers.
2.3 Research Approach
Jarvinen (2000) defines a research approach as “a set of research methods that can be
applied to similar research methods that can be applied to the similar research
objects and research questions.” In order to conduct research appropriately, there are
2 main research theories (inductive reasoning and deductive reasoning) can be
applied according to different research circumstances. These 2 research strategy can
be seen as seeking the truth from opposite directions (Walliman, 2001).
Deductive reasoning starts from more general to more specific (Trochim, 2006).
Trochim (2006) introduces that the deductive reasoning informally named a “top-
down” approach, which thinking up a theory as the first step. Then narrow down to
more believable hypothesis. Narrow down further until feedback observation to
address the hypothesis. The final step, confirmation, is testing the hypothesis with
specific data.
Figure 3: Deductive Reasoning
(Trochim, 2006, from http://www.socialresearchmethods.net/kb/dedind.htm)
Research Methodology Lihong Zhou
- 17 -
In contrast, inductive reasoning starts from more specific to more general, called
“bottom-up”. “Inductive research methodology begins with specific observations
and measures, begins to detect patterns and regularities, formulate some tentative
hypotheses, and finally ends up developing some general conclusions.” (Trochim,
2006)
Figure 4: Inductive Reasoning
(Trochim, 2006, from http://www.socialresearchmethods.net/kb/dedind.htm)
Nickerson (1986) states “effective reasoning requires the ability to develop
arguments and assess their validity to generate and test hypotheses, to judge the
plausibility of assertions, to identify possible courses of action, and to think through
consequences of choosing a particular course” (Nickerson , 1986, quoted in Watters
and English, 1995).
Liu (2002)’s research mentions inductive reasoning is more appropriate for studying
critical factors in information system project. Inductive reasoning starts from
problem statement and then generate conclusion from existing data which approach
uses current data to determine the key concept to be discuss. The main objective for
this dissertation is to generate a risk checklist for information system projects by
employing to research methodologies of literature review and case studies. In this
case, for this dissertation, it is better to use inductive reasoning as the primary
research approach.
Research Methodology Lihong Zhou
- 18 -
2.4 Data Collection and Analysis
There two kinds of data for sociological research according to McNeill (1990).
Primary data is captured by researchers at first hand mainly by surveys, interviews
and observations. Secondary data is researchers collected from some available
sources. Although, secondary data source might collected from primary data source
which is collected for its original specific purpose, secondary data can provide a
useful source to answer research questions and accomplish the research objectives
(Saunders, 2000). Data for this dissertation is only collected from secondary data
sources, mainly publications, academic journals, electronic source and official case
reports. No direct primary data collection activities are involved in this study.
There are several advantages of using secondary data, pointed out by Saunders et al.
(2000). The data is the enormous saving in resources, in particular researchers’ time
and money; the secondary data can be quickly collected, in addition, they might
higher-quality data than could be obtained by collecting by your own; secondary data
provide the only possibility of undertaking longitudinal studies for those research
projects with time constraints; re-analyzing secondary data can lead to unexpected
new discoveries; finally, secondary data provides a permanent and available source
can be easily accessed.
In contrast, Saunders et al. (2000) also point disadvantages for secondary data. Firstly,
secondary data might be collected not match the research purpose. Secondary data
employed in this dissertation is closely related to the topic of information systems
implementation or information systems failure. Secondly, Saunders et al. (2000)
indicates that secondary data may be difficult or difficult to access. Researcher for
this dissertation can conveniently access the data resource from university libraries,
university academic databases and journals, and Internet connections. Additionally,
aggregations and definitions may be unsuitable. This thesis strictly selected
secondary data, and criticized the suitability for this dissertation research. Finally, the
Research Methodology Lihong Zhou
- 19 -
initial purpose may affect the presentation of secondary data, especially for internal
organizational documents and external documents such as published company
reports and newspaper reports. This research project largely collects widely
published books, academic journals and official case reports, quit a few internal
organizational documents and newspaper reports are involved in the thesis.
There are 2 kinds of data for data analysis, quantitative data and qualitative data.
“Quantitative data refers to all such data and can be a product of all research
strategies. It can range from simple counts such as the frequency of occurrences to
more complex data (Saunders et al., 2000:326)”.
“Qualitative data are associated with such concepts and are characterized by their
richness and fullness based on your opportunity to explore a subject in as real a
manner as is possible (Robson, 1993, quoted in Saunders et al., 2000)”.
Differences between quantitative and qualitative data can be shown in the table
below: Quantitative Data Qualitative Data
Based on meanings derived from numbers Based on meanings expressed through words
Collections results in numerical and
standardized data
Collection results in non-standardized data
requiring classification into categories
Analysis conducted through the use of
diagrams and statistics
Analysis conducted through the use of
conceptualization
Figure 5: Quantitative VS Qualitative From (Saunders, 2000:380)
Research results from this dissertation are using qualitative data analysis
methodology from literature review and case studies.
Research Methodology Lihong Zhou
- 20 -
2.5 Dissertation Outline and Structure Map
After the introducing research background and methodology (Chapter 1 and Chapter
2), Chapter 3 stresses on information system implementation. It starts with defining
what is information system and clarifying the importance of information system
within organizations. This part describes how and why organzations need
information systems, and lists out how information systems can benefit organizations.
It ends with information systems implementation methodologies which provides
suggestions on how to actual develop an information system. Definitions and the
nature of failure are discussed in chapter 4, which gives an conception of information
failure by study the nature of failure. Chapter 5 introduces information project
management step by step. All major processes of information project management
with potential significant risks are discusses thoroughly. Chapter 6 dedicated to risk
management. This chapter not only illustrates nature of risk, but also identifies
features of risks in information system implementations. A major part of this chapter,
analyze how to use techniques of risk management to address and eliminate potential
risks.
Chapter 7 is an important component in this thesis, which chapter generates an
Information Systems Project Risk Checklist. The risk checklist is intened to record
all significant risks within information system projects based on literature reviews in
previous chapters. Chapter 8 and Chapter 9 implement the risk checklist onto INCIS
and LAS case studies respectively. 2 cases are experimentally evaluated in the form
of risk checklist, on the other hand, the risk checklist is significantly revised through
the case assessment. Chapter 10 is the main research result. The formal version of
Information System Project Risk Checklist is introduced with sufficient discussions.
All these Chapters can be shown in the structure map below:
Research Methodology Lihong Zhou
- 21 -
Figure 6:Dissertation Structure Map
This diagram can clearly illustrate the overall structure and processes in the thesis.
The research foundation and background is consisted of Chapter 3,4,5 and 6. The
main theory, Chapter 6 information system project risk checklist is introduced based
on the research foundation. Then, the risk checklist used as an analysis tool for
INCIS and LAS case studies in Chapter 8 and Chapter 9 respectively. Finally, and the
most importantly, research result, a revised version of information system project risk
checklist is announced in Chapter 10.
Information Systems Implementation Lihong Zhou
- 22 -
Chapter 3
Information Systems Implementation
Nowadays, information systems in organizations are playing a vitally central role.
Avision & Fitzgerald (2003) portrays that an information system can help an
organization operate more effectively and efficiently by providing useful processes
and information to its members and clients.
3.1 Definition of Information System
Information system can be comprehended on a number of distinct aspects.
Information system is an integrated, computer-based, user-machine system that
provides information for supporting operations and decision making functions (Awad,
1988). Sauer (1993) describes the information systems, particularly to computer-
based systems, as systems basically input and output information in order to
coordinate the work of many different organizational functions. Wolstenholme et al.
(1993) emphasize that information system has to present information and assist
information flow. Avison & Shah (1997) mentions an information system should not
only focus on the organization, but also the organizational environment. Information
technology (IT), Management information system (MIS), Decision support system
(DSS), Executive information systems (EIS), Strategic management information
system (SMIS) are considered as major components or functions for an information
system based on Robson (1997).
Figure 7: Information Systems (Developed from Robson, 1997)
Information Systems Implementation Lihong Zhou
- 23 -
3.2 Nature of Information Systems
The organizational information system, both computer-base or non-computer-based,
is built from an infrastructure. Based on Galliers (1992), the infrastructure groups the
organizational structure; communication channels (mails, telephones, faxes);
facilities (telephones, terminals, and computers); apparatus (PCs and servers);
software and necessary training.
Information systems are consisting of various components. Galliers (1992) explains
that in the real world, information systems are comprise from object, people, rules,
norms and commands. All systems, explained by Curtis and Cobham (1995), are
integrated from interrelated components with some purpose, goal and objective, and
designed processes within the system. The whole system processes can be showed as
the diagram below.
Figure 8: A model of system (Curtis and Cobham, 1995:15)
Robson (1997) develops a general conceptual model of information system based on
the model of system. The conceptual model simply exhibits several basic
components of an information system and map of information flow inside the system.
Information Systems Implementation Lihong Zhou
- 24 -
Figure 9: General of Information System (Robson, 1997:84)
Computer-based information systems, according to Stair and Reynolds (1999), are
composed of hardware, software, databases, communication media, people, and
procedures. “Hardware” mainly refers to computer equipments to perform inputting,
processing and outputting. “Software” is computer programs that control the
operation of the computer. “Database” is considered as the most important part of an
information system. A database stores all required and history of information.
“Communication media” is the electronic signal transmissions enable organizations
to link computer systems into effective networks. Information systems are purposed
to serve people, and a perfectly designed information system also needs personnel
who manage, run and maintain the system. Procedures are strategies, policies,
methods for using the information system. All factors of information systems, which
are explained above, can be shown in the Figure 10, made by O’Brien (2006).
Information Systems Implementation Lihong Zhou
- 25 -
Figure 10: Model of Computerized Information System (O’Brien, 2006)
Information systems are classified by Beynon-Davis (2002) along a horizontal
dimension or vertical dimension. Horizontal systems are differentiated by the type of
organization (private organizations and public sectors); Vertical systems are
distinguished by the three levels of human activity and decision made by the system.
Information Systems Implementation Lihong Zhou
- 26 -
3.3 Importance of Information Systems
High rate of information system failures has been shown at the beginning of the
article, but those astonishing numbers can’t stop numerous enterprises keep
implementing information system and ignore the great importance of information
systems. Current organizations heavily rely on information system in order to operate
efficiently and effectively (Yasin & Quigley, 1994).
Avison and Fitzgerald (2003) suggests that information systems are significantly
effect on current business which are said to be facing increased competition, global
challenges, and market shifts and rapid technological evolution. Boddy et al. (2005)
points out 3 main reasons of people more and more rely on information systems. In
the first place, “Electronic coordination”, modern information systems enable all
kinds of communications where information is routine. Information systems
electronically coordinate organizational activities of input, transformation and output.
Moreover, “Globalization”, information systems affect globalization by providing the
rapid flow of information which is extremely important for international distributed
organizations. “Information-intense firms”, information systems are particularly
important for those companies business in capture, create and distribute information
and knowledge to their customers.
Stair and Reynolds (1999) claim that information systems have the potential and the
ability to apply the latest technologies to work can result in a successful personal
career, organizations that reach their goals, and a society with a better quality of life.
The involvement of managers and decision makes in all aspects of information is a
major factor for organizational success, including higher profits and lower costs
(Stair and Reynolds, 1999).
Information Systems Implementation Lihong Zhou
- 27 -
3.4 Information Systems in Organizations
It is a well known truth that the success of an organization depends on its information
systems (Beynon-Davies, 2002). Information systems are employed by business
organizations for several intentions: to restructure and revise the organization
composition to the form of achieving their goals; to improve customer service and
customer satisfaction (Stair & Reynolds, 1999). Stair & Reynolds (1999) portrays
that the information technology once has been used to the organization, the nature of
work and the shape of organization have been changed.
An organization is a formal collection of people and other resources established to
accomplish a set of goals, according to Stair & Reynolds (1999). Beynon-Davies
(2002) points out there are 2 perspectives bottom-up or top-down. The definition of
bottom-up structure indicates that an organization is built up by amount of human
beings and their actions. A top-down structured organization means the
organizational structure exists independently of the human belonging to it. The top-
down concept considers that a larger social structure direct or constrain human
actions. Another important theory the structure theory is created by Anthony Giddens
(Beynon-Davies, 2002). This theory presents a “duality of structure”, which claims
that the human actions produce and reproduce social structure, on the other hand, the
social structure limits human actions from using the structure as a resource in
translating their own and other people’s actions. The duality of structure can be
exhibited by the Figure 11 below:
Figure 11: Duality of Structure (Beynon-Davies, 2002: 220)
Information Systems Implementation Lihong Zhou
- 28 -
Organizations can also be classified as for-profit organizations and nonprofit
organizations, according to Stair & Reynolds (1999). The main purpose for for-profit
organization is to enlarge incomes by maximizing revenues and minimizing costs.
Nonprofit organizations don’t take profit as the primary goal, such as social groups,
religious communities, and universities. An organization is a system regularly using
money, human, materials, machines and equipments as resources (Stair & Reynolds,
1999).
Figure 12: The elements and context of a computer-based information system
(Boddy et al., 2005)
From another point of view, Boddy et al.(2005) describes that the organization is one
of three social elements (people; procedures; organization). People are the most
sophisticated elements. It includes staffs who input data and who analyze outputs of
information system, and managers who administrate full scale of activities;
Information Systems Implementation Lihong Zhou
- 29 -
procedures are the rules or restricted actions of how people interact with the
information system; the organizational context is consisted procedures and human
actions of interacting with the system. The results of an information system depend
on the interaction between people, system and context (Boddy et al., 2005). The
Figure 12 above illustrates how organizational context working on an information
system project.
Information Systems Implementation Lihong Zhou
- 30 -
3.5 Information Systems Development Methodologies
To make information system actually take place, various different processes and
activities are required (Flynn, 1992). Flynn (1992) claims that to develop a normal
and simplest system, a project team is the basic human resource and other
requirements as hardware systems, supporting software platforms and
communication networks; besides, users and computer specialists cooperatively list
out user requirements and a system specification, select and integrate computer
hardware, conduct system coding, test system and user training. The series of
designing activities and developing forms are generally known as systems develop
process.
Information systems development is a fatal organizational process for many
organizations (Beynon-Davis, 2002). It is essential for an information systems
development follow a confirmed methodology to ensure the systems are conceived,
analyzed, designed and implemented. The conception of information systems
development methodology is defined, by Avison & Fitzgerald (2003), as:
“A collection of procedures, techniques, tools, and documentation which will help
the systems developers implement a new information system. A methodology will
consist of phases and sub-phases. These sub-phases provide developers plenty of
opportunities of selecting the most appropriate techniques at each stage and
effectively supporting developers to plan, manage, control, and test the project.”
(Avison & Fitzgerald, 2003: 20)
Methods, techniques and tools are pre-requisite for project producers when they
undertaking the project process (Avison & Fitzgerald, 2003):
Information Systems Implementation Lihong Zhou
- 31 -
“Methods”: these build up the developing frameworks (Beynon-Davis, 2002).
Beynon-Davis (2002) identifies three major types of developing methods:
1. Structured methods, with a linear model of the development process and clear
identification of inputs and outputs for each phases. Certain techniques are equipped
within structure frameworks.
2. Rapid application development (RAD) methods.
“This kind of methods employs an iterative model of the development process and
generally specifies high level phases based around some form of prototyping.”
(Beynon-Davis, 2002: 323)
3. Object-oriented (OO) methods. Such methods use object attempt to simulate the
real world and build up the model by identifying objects and attributes, relationships
for these models. Object behaviors are also defined in OO methods.
“Techniques”: mainly relate to analyzing and designing phases of developing; they
are usually components of methods (Beynon-Davis, 2002). A technique is a solution
of a particular activity in system development process (Avison & Fitzgerald, 2003).
Beynon-Davis (2002) identifies 2 distinct groups of techniques, developer-centric
techniques and user-centric techniques.
Developer-centric techniques are especially designed for developers to understand,
document and communicate problems to other developers. These techniques include
data analysis techniques, process analysis techniques and object analysis techniques
(Beynon-Davis, 2002).
User-centric techniques support project developers understand some work
environment and the potential technologies can be used by the development. These
techniques include prototyping, scenarios and use cases (Beynon-Davis, 2002).
Information Systems Implementation Lihong Zhou
- 32 -
“Tools”: artifacts used in the system project, include hardware, software, data and
internal, or external, communication facilities (Avison & Fitzgerald, 2003), (Beynon-
Davis, 2002). These tools are graphical user interface tools, fourth generation
languages, transaction processing systems, database management systems, local area
and wide area communication tools.
Nature Of Failure Lihong Zhou
- 33 -
Chapter 4
Nature of Failure
Last chapter illustrates basic conceptions and nature about information system. As
the next step, to study Information systems failures, it is essential to understand the
definitions of failure and the nature of information systems failures.
4.1 Definition of Failure
All kinds of technological and organizational systems suffer from failures (Sauer,
1993). As the extremely fundamental conception, there is no commonly agreed
definition of the definition of failure.
Oxford Advanced Learner’s Dictionary of Current English (1991) defines the success
as “achievement of a desired end, or of fame, wealth or social position.” In the
contrast, the dictionary portrays the failure as “failure of success (for instance: failure
in a examination); state of being inadequate; not functioning as is expected or
required.”
Fortune and Peters (1995) claims that “failure is something that has gone wrong or
not lived up to expectations.” (Fortune and Peters, 1995:21)
Fortune and Peters (1995) argues that almost all judgement about failure are
subjective; those comments about failure are coloured by personal perception,
circumstances and expectations. However, in several situations, it is quite hard to
define a project or a system is successful or unsuccessful. And distinct people may
have different arguments and opinions about a particular project, which some might
think it is a total failure but some people think it is acceptable.
Nature Of Failure Lihong Zhou
- 34 -
4.2 Types of Failure
Fortune &Peters(1995) suggests a 4-type of failure on the perspective of how well a
project satisfies predefined project objects and user requirements. The 4-type of
failure is shown in the Figure 13 below:
Figure 13: Failure Types (Fortune &Peters, 1995: 23)
Type 1 failure stands for objectives from designer, sponsors or users are not fully
met. Those projects met original objects but with inappropriate or undesirable
consequences and side effects are categorized into failure type 2. Some failures are
designed to fail at particular time or occasions, which kind of failures can be seen as
an important part of the integrated system as the reason of safety issues. This kind of
failures are failure Type 3. The final kinds of failure, type 4, is when the project
meets objectives and requirement without undesirable consequecies or side effect,
benefits and merits of the project is no longer exist. (Fortune &Peters, 1995)
4.3 Information Systems Failures
Sauer (1993) indicates that information systems are the product of a process which is
open to flaws. Beynon-Davies (1999) not just illustrates similar statements but also
distinguishes the concepts of flaw and failure. Compare to concepts of failures
introduced in last section, flaws can be described as perceptions of undesirable
situation faced by project teams and stakeholder which can be solved at a cost or
accepted at a cost.
Failure Types Type 1: Objectives not met Type 2: Undesirable side effects Type 3: Designed failures Type 4: Inappropriate objectives
Nature Of Failure Lihong Zhou
- 35 -
There is a broad spectrum of failures, Sauer (1993) claims that each type of failure of
technological and organizational system can be studied separately within its own
field, so as information system disasters has its unique features. Information system
failures, are multi-causal particularly for those complex projects (Bignell & Fortune,
1984), usually involve a complex interaction of factors within the human and
technical issues set in various situations rather than directly failure of a single man
(Sauer, 1993). Sauer (1993) lists determinants of information systems success/failure
from previous studies
Russell Ackoff (1967) Designers don’t understand their
objectives; Dearden (1972) Incompetent or ineffective for system
developers; Morgan and Soden (1973) Senior Information System Executive as
the prime determinant of success or failure;
Argyris (1971) Executive resistance as the reason of lack of interpersonal communication skills;
Colton (1972) Lucas (1975) Boland and Hirschheim (1985)
Behavior and social oriented problems are hindering projects development.
Figure 14: Determinants of Information systems Success/failure
Quoted in (Sauer, 1993: 21-22)
Laudon and Laudon (1991) classify information systems development problems into
4 areas: Design, Cost, Operations, Data. Lyytinen and Hirschheim (1987) proposed 4
concepts of information system failure: Correspondence failure; Process failure;
Interaction failure, and Expectation failure. Sauer (1993) explain these 4 factors as
follow:
Correspondence failure stands for failures unable correspond to predefined
objectives and requirements. This is the most common kind of information system
failure. Beynon-Davies (1999) indicates that an evaluation is required to be
conducted during the project, if the mismatch of correpondence between original
objectives and the evaluation, the project is considered a failure.
Nature Of Failure Lihong Zhou
- 36 -
There are two distinct kinds of Process failures: fail to produce a system at all, and
an information system is produced but fail to meet the predefined or reasonable
constraints of budget and time.
Interaction failure stresses on user satisfaction. It is very common that systems
successfully implemented but fail to satisfy its users. If a system is heavily used
which means the system is a success; in the contrast, if the system is barely used or
working with serious problem which means the system is a failure.
Expectation failure is a conclusion of 3 previous failures, which is defined as an
information system unable to achieve the expectations from a specific stakeholder
group.
Project Management Lihong Zhou
- 37 -
Chapter 5
Project Management
Project management provides people with a pool of powerful tools that improves
their ability to plan, implement, and achieve specific organizational objectives.
Moreover, project management is more than just a set of tools, it is also a
management style of results-oriented that can buildup collaborative working
relationships among various characters (Gray & Larson, 2006). Avison & Fitzgerald
(2003) indicate that many information system failures may reason from inadequate
project management.
“A project is a complex, non-routine, one-time effort limited by time, budget,
resources, and performance specifications designed to meet customer needs (Gray &
Larson, 2006:4).” All projects generally follow a project life cycle. Project
management is not just a management tool, but also a fundamental discipline of
doing business. Gray & Larson (2006) illustrate that the increase of the importance
and the role of the project management in contributing to the strategic direction of
organizations.
The project life cycle model presented by Gray & Larson (2006), which separate the
entire life span of a project into 4 sequential stages: defining, planning, executing and
delivering. On the defining stages, project mangers have to detailed plan the whole
project, such as define project scope and objectives, project team construction and
assigning major responsibilities to appropriate personnel. Planning stage determines
what the project will do, when it will be scheduled, whom it will benefit, how to
execute the quality control and initiate the budget planning. Executing stage occupies
the major portion of the project work. On this stage, factors of time, cost, and
specification are used for steering the project processes. As the final stage, the
Project Management Lihong Zhou
- 38 -
delivering stage includes 2 tasks: delivering the project product to the customer and
redeploying project resources.
5.1 Managing Information Systems Projects
Project management techniques are also the safeguard of keeping information
systems projects from failure. Avison & Fitzgerald (2003) depict that project
management in information systems projects is a theme that is important in most
methodologies; it can efficiently and effectively managing the project development,
delivering the project on time, within budget, complete and with the highest quality.
The basic intention of managing information systems projects is to develop required
systems that are (University of Sheffield, 2000):
(a) On time
(b) Within budget
(c) To a quality standard
(d) Modifiable to the future improvement
(e) Within the limits of the resources
Project management in information systems development projects is regarded more
difficult than other projects, according to University of Sheffield (2000). Projects in
constructing industry have much clearer project objectives and milestones. In
contrast, objectives for information systems projects are usually more abstractive and
have more ambiguous milestones. Besides, an information system project team is
constructed from people with various backgrounds. University of Sheffield (2000)
indicates that project managers have to manage multi-disciplinary team of people
who may never worked together before and may have different opinion with each
other’s point of view. Additionally, more sophisticated software and hardware are
normally required which demand system developers constantly changing developing
tools and techniques, and also not easy for final users to accept the advanced
technology.
Project Management Lihong Zhou
- 39 -
On the basis of several references (Yeates & Cadle, 1996); (Beynon-Davies, 2002);
(Avison & Fitzgerald, 2003); (University of Sheffield, 2000), the thesis categorizes
the complete information systems project management processes generally into 3
phases:
(1) System planning and estimating;
(2) System design, development and implementation;
(3) System testing and maintenance;
At the beginning stage, planning and estimating, project scope and objectives are
defined; feasibility study and strategic planning are also conducted in this phase of
project. Besides, it is essential for a project manager to buildup a powerful project
team and to choose an appropriate developing methodology. At the stage, after
braking down the entire project and main tasks into several iterative sub-tasks,
project team needs to estimate systems development resource including the duration
and size of the system, potential problems and risks, and costs.
In the system development, design and implementation stage, project team programs
and implements the information system, as planned from the last stage, with certain
quality assurance plans. As the last phase of the whole project, testing and
maintenance are also vitally important. Testing is a tedious and exhausted work, but
it is compulsory. After thoroughly system testing, some of the defects detected are
corrected. University of Sheffield (2000) describes that in reality, as the reason of
several changes are considered to be necessary, parts of the development process
might repeated in system maintenance.
All these 3 major stages of information systems project management are detailed
introduced and discussed in the following parts, as well as necessary project
management tools and techniques in each stage.
Project Management Lihong Zhou
- 40 -
5.2 System planning and estimating
As the very starting stage, it is the foundation of the entire project, and this stage is
critically important of associating to the project’s success/failure. Yeates & Cadle
(1996) state that first stage is important as its where the foundation of the whole
project are laid; future works are build based upon this foundation, thus the
importance of the stage should not be underestimated.
Yeates & Cadle (1996) clams that there are 5 concepts that the project team has to
define:
What is be carried out?
Why is it being carried out?
Who is going to do it?
How is it to be carried out?
When is it to be carried out?
In the first place, project team has to perfectly define and understand user
requirement. On the basis of user requirement, project objectives, scope and potential
constraints needs to be clearly identified. Project team can be supported from other
source from feasibility study report, project brief, project terms of refer-source and
contract document (Yeates & Cadle, 1996), on-site observation and interviews.
Yeates & Cadle (1996) states that a starting point for a good plan is proper
understanding of the user requirement, and the project manager must ensure user
requirements are fully comprehended before planning begins. An important problem
pointed out by University of Sheffield (2000) which states that customers usually
miscomprehend between “what we need” and “What we want”. It is common that an
organization consider a system which could be completely different from the system
design from system developers. In this case, system developers have to redesign both
of the hardware and the software in order to achieve the overall objectives.
Project Management Lihong Zhou
- 41 -
The next step is to estimate the overall costs of developing and maintaining the
system as well as how the organization will be benefited from the expected
information system. Underestimated project costs and over-expected project benefits
could be a reason of project failure and customers’ dissatisfaction. Yeates & Cadle
(1996) suggest that each project needs to have a Business Case which sets out the
main problems or opportunities which are to be addressed; business case also
contains costs budget and expected profits.
A complete and effective project team is another key of the project. University of
Sheffield (1996) points out project team members are carefully selected for their
competency and compatibility; the project manager is responsible for managing time,
cost, resource and accurately communicates to every team member. The project team
also needs to set up goals for productivity, and team members must be motivated to
achieve them. Additionally, Yeates & Cadle (1996) mention a particularly important
factor that the roles and responsibilities are clearly set out in the project.
The basic method of planning and estimating are breaking down the whole project
into various stages. The decisions will be made on who is going to in charge for what
tasks, how to finish each task and when it has to be finished. Work breakdown
structure and Gantt Chart are the basic tools of planning and controlling the project.
(University of Sheffield, 2000)
Work breakdown structure (WBS) is a traditional approach and has been widely used
in many industries for a long time (Yeates &Cadle, 1996). Yeates & Cadle (1996)
introduce the concept of work breakdown structure as:
“Work breakdown structure is to break the overall project down progressively into
smaller and smaller chunks until we end up with individual tasks, or work packages,
that we can estimate sensibly and assign to team members.” (Yeates & Cadle,
1996:74)
Project Management Lihong Zhou
- 42 -
The WBS diagram of an information systems development example is shown in the
appendix II.
Gantt chart, according to University of Sheffield (2000), is named after its originator
Henry Gantt. Gantt chart is a visual tool for communicating schedules to the
organizations. On the left hand side of Gantt chart, there are tasks and durations for
each task. A bar represents the timing and duration on the right hand side of the chart.
It can be used to see which tasks run in parallel and to calculate the total duration. An
example of Gantt chart is shown as follow:
Figure 15: An example of Gantt chart from University of Sheffield (2000),Page 2
Yeates &Cadles (1996) point out that effective planning is essential to the successful
execution of an information systems project. In Sumner (2000) conclusion, risk
factors of formal information management projects and point out 8 out of 18 risk
factors are related to this stage of project initiation. Thus, project team is required
more attentions on this part of development.
Project Management Lihong Zhou
- 43 -
5.3 System Design, Development and Implementation
This stage is all about system designing and implementation of the final system to
the organization. Obviously, the project must be finished within planned duration and
budget, but another important factor is to ensure the final system with certain
required quality.
The quality of an information system can be evaluated by numbers of defects, faults
or bugs, failures and errors. According to University of Sheffield (2000), a defect is
an omission, imperfection, ambiguity or fault with a product; a failure is a
breakdown of a project or productions with unsatisfactory results. University of
Sheffield (2000) indicates that Verification and Validation (V&V) usually used for
checking the quality of the project. Validation refers to checking that the system
conforms to its specifications; validation refers to checking system achieves
customers requirements (Sommerville, 1996).
Yeates and Cadle (1996) present a model provided by the European Foundation for
Quality Management (EFQM). The model provides a group of quantitative
evaluation of the key criteria for a total quality business.
Figure 16: The Business Excellence Model of EFQM (Yeates and Cadle, 1996:171)
Project Management Lihong Zhou
- 44 -
The model illustrates relationships between people, processes and results. A
successful Business Results is come from People Satisfaction, Customer Satisfaction,
Impact on Society and Impact on Society. These 3 factors depend on the Leadership
of driving factors of People Management, Policy and Strategy, Resources and
Processes. The Enabler concerns with how the organization approaches each of these
factors. The results represent what the organization is achieving or has achieved. The
percentage for every factor is the relative value.
There are 7 approaches of main tasks and activities of assuring information system
quality, based on University of Sheffield (2000)’s conclusion from Pressman
(1987/1992):
1. Application of Technical Methods
2. Formal Technical Reviews
3. Software Testing
4. Enforcement of Standards and Procedures
5. Control of Change
6. Measurement
7. Record-keeping and reporting
System design is another critical issue in this part of project. University of Sheffield
(2000) illustrates unable of fulfilling the customer expectation and achieving basic
business requirement mainly are resulted of the poor system design. Additionally,
based on the research from Kasser (1998), requirements change from customer
during system designing or after is the most dangerous factor from 33 critical factors
of information system failure.
Project Management Lihong Zhou
- 45 -
5.4 System Implementation, Testing and Maintenance
After detailed system design, system implementation comes next to the customer’s
agreement on the systems specification, which shows a complete system description
and other specific project related information. Curtis and Cobham (2002) identify
several related tasks that need to be finished in system implementation. These tasks
are writing and testing programs, acquiring and installing hardware, training final
system users, changeover data from old manual or computerized information system
to new database. It is important that they are all completed before actual changeover
to the new system, and some of these activities can be carried out simultaneously.
System implementation can be divided into software system implementation and
hardware system implementation. University of Sheffield (2000) illustrates new files
need to be ready before the new system installation, and the system changeover
requires no significant changes in the data used. Testing is an indispensable but tedious activity during the system implementation.
Testing is a basic approach of ensures system with certain quality and ensuring user
requirements are achieved. Testing work must also be detailed planned with suitable
testing strategies. University of Sheffield (2000) provides 3 testing strategies: top-
down, bottom-up and mixed.
According to Curtis and Cobham (2002), there are 4 types of system changeover.
University of Sheffield points out advantages and disadvantages for each system
conversion strategy:
1) Direct Changeover: New systems replace the old legacy systems entirely. The
advantage of this approach is being quick. However, this approach heavily relies
on liability of the system design, implementation and well trained staffs.
Advantage: the organization can get immediate benefits from the new system.
Disadvantage: High risk.
Project Management Lihong Zhou
- 46 -
2) Parallel Changeover: This approach is commonly used if the new system and
old legacy systems are quite similar. This approach requires the old system and
new system run in parallel.
Advantage: this approach can ensure the organization processes naturally without
any pitfalls during changing over the old system to the new system.
Disadvantage: expensive because of both of old and new systems are required
operate simultaneously.
3) Modular Changeover: This approach is for those systems can be separate up to
several identical modules and replace them piece by piece in parallel.
Advantages: every module is fully tested; users can easily accept the new system
and familiar the new system.
Disadvantages: Special attentions must be paid on making modules work
cooperatively.
4) Phased Changeover: If a system is constructed from numbers of individual
modules, the old system can be slowly phase out, in the mean time the new
system gradually phase in.
Advantages: problems exposed at the implementing phase can be learned and
avoided for the next phase.
Disadvantages: requires a long period of time which can create both positive and
negative user problems.
5) Pilot Changeover: Before the final conversion, project team provides a prototype
to small group of users. Project team continuously revises the system from users’
feedbacks.
Advantages: the final system is fully tested and achieved user requirements are
ensured.
Disadvantages: need users’ participation and may give customers an impression
that the new system is unreliable.
Project Management Lihong Zhou
- 47 -
Maintenance is taken as the last step of system developing and last phase of System
Life Cycle. Maintenance is a long non-stopping process which continuously modifies
and improves the organizational information system. Although all information
systems are examined by detailed testing, University of Sheffield (2000) claims that
not all defects can be detected by the testing and some of those bugs can only be
emerged after certain duration of system operating. System maintenance includes
hardware maintenance and software maintenance, based on Curtis and Cobham
(2002). Hardware maintenance mainly concerns on maintaining system equipments
such as PCs and network. Software maintenance is a complex and costly process
(Curtis and Cobham, 2002). University of Sheffield (2000) defines software
maintenance is substantially rewriting the original software program.
Risk Management Lihong Zhou
- 48 -
Chapter 6
Risk Management
In Frosdick’s survey, the definition of risk is portrayed as “taking account of both
gains and losses; the probability that a particular adverse event occurs during a
stated period of time, or result from a particular challenge (Royal Society, 1992); a
combination of the probability, or frequency, of occurrence of a defined hazard and
the magnitude of the consequences of the occurrence (British Standard Institution,
1991).” Wright and Wright (2001) suggest that many researchers proposed that risks
maybe the reason of ERP failures. Risk management, the basic purpose, is to
significantly improve project quality and project performance by using
methodologies of systematic identification, evaluate and manage potential project
risks (Chapman & Ward, 1997). Risk management has became a main part in
organizational activities and it also help all activities achieve their objectives directly
and efficiently (Tchankova, 2002). Besides, risk management provides decision
makers systematic approaches to dealing with risks and uncertainties (Williams et al.,
2006). Yeates and Cadle (1996) conclude that projects of information systems are
becoming increasingly complex, and more unpredictable risks are exposed in these
projects. Although it is difficult to avoid risks all together, risks can be identified and
dealt with their negative impact by avoiding or mitigating (Yeates and Cadle, 1996).
As the risks are regarded as the main reason of information system failure and the
high unsuccessful information system project rate, the great necessity of risk
management is obvious and inevitable. Bandyopadhyay et al. (1999) not just point
out risk management for information projects is one of the most important tasks for
project developers, but also list 3 purposes of risk management in information
systems projects:
Risk Management Lihong Zhou
- 49 -
(1) Protect IT assets such as data, hardware and software from external threats.
(2) Avoid or mitigate complete losses by selecting and implementing the best
combination of security measures.
(3) Protect project from internal threats such as technical failures, sabotage and
unauthorized access.
According to Williams et al. (2006), before developing the formal risk management
plan, it is fatal to comprehend potential organizational risks, basically there are 3
classifications of risks: risks must to be managed, such as risks may jeopardize
project quality, government requirement and environment related issues; internal and
external classic risks; the third group of risks are those have no mandatory
requirements to work on, which usually without external forcing compliance and no
clear self-preservation reason to manage them.
Chapman & Ward (1997) present that risk management involves evaluating risks and
opportunities; assess and develop a contingency plan supporting the base plan, which
exhibits objectives of the project and how to achieve them. There are 2 main
approaches of risk management. As the main approach of risk management,
proactive risk management concerns planning which proactively identify potential
future threats. University of Sheffield (2002) coins the proactive risk management is
the best practice in Information Systems industry of preventing unexpected
consequences. Reactive risk management plan acts as a complementary approach in
the real risk management. This approach focuses on strategic plans on how to react
after a crisis.
Risk Management Lihong Zhou
- 50 -
6.1 Risk Management Process
Three main risk management stages are identified by Williams et al. (2006): (1) risk
recognition and identification; (2) risk assessment and prioritization; (3) risk
management. A general risk management process is displayed by Yeates and Cadle
(1996):
Figure 17: The Risk Management Process (Yeates and Cadle, 1996: 190)
A similar but more thorough risk management process made by Standards Australia
and Standards New Zealand is shown by Williams et al. (2006), and the process
diagram is shown as follow.
Figure 18: Risk assessment process
(Standards Australia and Standards New Zealand 2004 a, b)
(Williams et al., 2006)
Risk Management Lihong Zhou
- 51 -
Establish the context and risk identifications are 2 main tasks of risk recognition, or
risk assessment. Context establishment defines what is at risk, and then risk
identification points out uncertain events potentially jeopardize or benefits within the
context. Possible consequences of these uncertainties are defined in the phase of risk
identification. Risk prioritization aims to comprehend the nature and level of risks
identified in the stage of risk recognition. The first step of risk prioritization is risk
analysis which defines levels of risks by likelihood (probability and frequency of risk
occurrence) and risk consequences. After analyzing risks, risks are evaluated and
ranked by a certain risk-acceptance criterion. The next stage is to manage risk by
various methodologies (Williams et al., 2006). The simplest way of dealing with
unacceptable risks, claimed by Williams et al. (2006), is “four Ts” presented by
European Foundation for Quality Management (2005). “Four Ts” are Terminate (stop
activities associated to the risk); Treat (develop a plan to deal the risk); Tolerate
(accept the risk with undesirable defects); Transfer (transfer negative impact of risk
to other entity or organizations).
6.2 Risk Identification
As the very first step of risk management, risk identification is required to examine
the entire project and identify potential risk area. Risk identification initiates by
attempting create a list of all possible risks that could affect the project (Gray &
Larson, 2006).
Frosdick (1997) claims the brainstorming is the main intuitive technique of risk
identifying, which involves a group generating ideas off the top of their heads. Gray
& Larson (2006) briefly introduces the method. Typically, in the planning phase, the
project manager organize a risk management team consists of core developers and
relevant stakeholders. Brainstorming participants are encouraged to keep an open
mind and generate as many possible risks as possible.
Risk Management Lihong Zhou
- 52 -
Besides, according to Frosdick (1997), inductive risk identification analysis includes
preliminary hazard analysis, checklists and human error analysis. Hazard and
operability studies (HAZOPS) and Failure modes and effects criticality analysis
(FMECA) are the most common techniques. The HAZOPS is structured
brainstorming exercises for a multi-disciplinary team of experts systematically
consider every item in the project. Possible deviations from the intentions are
detected by using guidewords such as “not”, “more”, and “less”. FMECA is based on
thoroughly understand the whole organization and project after investigation, and
this technique usually carried out individually. FMECA presents a step by step
process for examining design weakness and possible failure causations base on either
hardware orientation, focusing on potential hardware equipment failure, or event
orientation, emphasizing on unexpected outputs and the negative effects.
Each project is unique, and various risks are unpredictable. It is obvious that in some
particular area, risk could arise and it is difficult for a project manage ensure all
possible risks are pointed out (Yeates & Cadle, 1996). Yeates & Cadle (1996) suggest
the project manager to be strictly honest about the risks.
Kliem & Ludin (2000) highlight the purpose of risk identification are provides
project managers to select personnel who are qualified of risk management processes,
identify critical risks and highlight potential risk areas, advanced understand the
system and processes of the project and its risks or goals.
6.3 Risk Assessment and Management
Activities of risk assessment, impact consequences and likelihood, are carried out
next to identifying and describing risks. Risk assessment is a fundamental decision
making process resulting in the selection of appropriate safeguards for information
systems projects (Lichtenstein, 1996). Lichtenstein (1996) defines 2 steps of risk
assessment. For the first step, risk analysis processes define the scope of the risk
Risk Management Lihong Zhou
- 53 -
assessment; identify information resources and prioritizes risks. The second step
involves a risk management process makes decisions of unacceptable risks; risk
transferring processes, or ignore risks, or reduce impacts from risks.
It is impossible to eliminate all risks completely but a better way is to reduce risk
likelihood and risk impact. The risk impacts largely depend on proportions of the
programming work undertaking by project developers based on Yeates & Cadle
(1996). The risk impact will be less if only a small proportion of programming for
coding team. Another factor of risk evaluation, mentioned by Yeates & Cadle (1996),
is the likelihood or probability of the risk context. There is no certain mathematical
method to value risk impact and likelihood, but they can be measured by the scale
below.
Risk Impact:
Figure 19: Risk Impact Scale (Developed from Yeates & Cadle (1996))
Risk Likelihood:
Figure 20: Risk Likelihood Scale (Developed from Yeates & Cadle (1996))
On the basis of 2 scales, a risk analysis formula is presented by University of
Sheffield (2001):
Risk Exposure = P × L
P: risk likelihood; L: risk impact;
Risk Management Lihong Zhou
- 54 -
The definition of risk exposure, provided by Boehm & DeMarco (1991), is the
probability of the risk occurrence and the potential loss caused by the risk.
After risk identification and risk analysis, there are 2 basic approaches of dealing
with the risks provided by Yeates and Cadle (1996). The first approach is to attempt
to prevent the risk from occurring, called avoidance. Mitigation is the other approach,
which intends to reduce the impact after risks taking place. In practice, these 2
approaches are both considered as the avoidance approach may fail and not as
efficient as it is expected (Yeates and Cadle, 1996).
Risk management is an ongoing nonstop process and more unexpected risks might
expose as the project developing. Yeate & Cadle (1996) suggest that the project
manage team should documents a risk management plan, particularly for those large
projects with a large amount of complex risks. The risk management plan should
enclose a statement of the scope and intensity of the risk management; an
explanation of the risk management cycle about when and how will carry out
particular risk treatment; a human resource plan about risk management; regularly
report of risk management status to senior managers.
Information Systems Project Risk Checklist Lihong Zhou
- 55 -
Chapter 7
Information Systems Project Risk Checklist
Current studies about risk assessment in information systems development are
introduced in the Introduction chapter, which chapter also points out the significant
insufficiency of current researches. In former chapters, the dissertation reviews and
discusses background fundamental definitions and theories of nature of information
systems, nature of failure, project management methodologies and techniques, risk
management on the aspect of information systems projects. The main objective of
this chapter, as well the main purposes of this thesis, is to frame an information
systems project risk checklist. All possible major risks that have been considered as
main causations of information system disasters in previous publications are intended
to be recorded in this risk checklist and discussed afterward. The purpose of
constructing this checklist is to produce an elementary guideline for information
systems project developers in order to avoid potential risks and keep the project on
the right track.
7.1 Literature Review Based Checklist Building
The initial version of Information Systems Project Risk Checklist, in this chapter, is
generated from deeply literature review on the aspect of risk management on
information systems and covers most frequent risk factors in information systems
project failures from previous researches.
The second version, the formal version, is presented at the end of case studies, is a
revised edition by applying the checklist onto the case study of INCIS and London
Ambulance Service.
Information Systems Project Risk Checklist Lihong Zhou
- 56 -
Figure 21: Diagram of revisions on Information Systems Project Risk Checklist
Generally, both checklists are consists of 5 components, namely Pre-project, Project
management, Customer, Project management, Project technical aspect, Project
development. Relations between these 5 parts can be presented as the picture below:
Figure 22: Relations of 5 components
from Information Systems Project Risk Checklist
All potential information systems project risks are categorized into 5 facets which are
illustrated on 5 separated lists. Various risks are simultaneously recorded on 2 or
more lists, consequently 5 lists are closely interconnected.
Information Systems Project Risk Checklist Lihong Zhou
- 57 -
7.2 Pre-Project
This part of checklist concerns critical issues that project team needs to beware
before actual project designing. This part not only covers project team side of story
but also identifies some negative issues within customers’ organizations, and the
most importantly, some risks about contracts are also illustrated. Information Systems Project Risk Checklist Part 1of 5
Pre-Project 1 Complex and unclear relationship among partners, customers and suppliers 2 Disagreement between contract involved partners 3 Failure to consider existing relationships when replacing systems 4 Failure to consider existing systems when replacing systems 5 Lack of senior management support 6 Project scope, objective and requirement specifications are ill defined 7 Resources are not allocated well 8 The customer lack of formal experience 9 Unclear payment schedule without linked milestones 10 Unsuitable business plan and vision 11 Without backup plans for delay or under-performance
As introduced in Chapter 5.1, before the project, the customer needs to have a
general understanding of a sound project plan and appropriate user requirement
specifications. Any modifications, or even revision, after contract on project user
requirements are risky (University of Sheffield, 2000; Sumner, 2000; Kasser, 1998).
Yeates & Cadle (1996) claims ambiguous roles of, and vague relationships with
every project parties, in addition of internal political difficulties in the customer’s
organization are major risk factors in information system projects. Customer side of
management plays an important role in this phase. First of all, supports from
management are the basic requirement of the project. Nah et al. (2001) suggest that
top management support is needed throughout the implementation and top
management needs to publicly and explicitly identify the project as a top priority.
Based on Kasser (1998)’s study, lack of senior management support is one of top 10
risks. With management’s support, adequate resources (fund, staff, etc) can be
allocated to the project. Yeates and Cadle (1996) imply that the contract is the biggest
Information Systems Project Risk Checklist Lihong Zhou
- 58 -
risk since the contract is a strategic tool for parties convey potential project risks,
particularly project scope is ill-defined or not agreed between parties. The contract
needs to be suitable for the specific project with clearly defined payment schedule
and backup plans in case of delay and other unexpected emergencies. A clear
business plan and vision to steer the direction of the project is needed throughout the
project life cycle (Buckhout et al., 1999, quote by Nah et al., 2001). A business plan
that outlines proposed strategic and tangible benefits, resources, costs, risks and
timeline is critical (Wee, 2000 quoted by Nah et al., 2001).
7.3 Customer
Customers might not familiar with information technology. They might unable to
fully understand certain specific risks involved in information systems projects. Here
are some risks and suggestions on the aspect of customer. Information Systems Project Risk Checklist Part 2of 5
Customer 1 Changing requirements, scope and objectives 2 Conflicts between user departments 3 Final Users are unfamiliar with the technology and require additional training 4 Internal political difficulties 5 Lack of customer support 6 Reluctance of changing environment to accept the new system 7 Resources are not well assigned to the project team 8 Unable of unifying customers perspective on the project
Information systems projects require fully cooperation from all involved parties.
Conflicts between user departments and internal political difficulties can bring
troubles to information systems implementation (Yeates and Cadle, 1996). With the
development of information system, organization structure and culture will be largely
changed which may lead to significant user resistance and cause a series of risks.
Lyytinen (1988) illustrates that a stakeholders expectation failure is a gap between
stakeholders’ expectation in some ideal or standard and actual system performance. It
is critical to unify involved parties’ perspective on the project.
Information Systems Project Risk Checklist Lihong Zhou
- 59 -
7.4 Project Management
As illustrated in the Chapter 5 of Project Management, project management provides
efficient and effective tools to manage information projects, assists project team
eventually successfully implement the information system. However, there are also
specific rules about project management. Information Systems Project Risk Checklist Part 3of 5
Project Management 1 Lack of an effective methodology and poor estimation 2 Unrealistic schedules and budgets 3 Lack of effective risk controlling plan 4 Milestones and related deliverables are ill-defined 5 Inexperienced team member in the business or technology 6 Inappropriate structure of project team 7 Team members are not fully committed to the project 8 Too many junior staff or too many senior staff 9 Unfamiliar environment for system developers 10 Lack of process and standard 11 No single person is accountable/responsible for the project 12 Unrealistic deadlines 13 Inappropriate staffing, personnel shortfalls
Yeates and Cadle (1996) suggests that an initial project plan may not necessarily be
presented before the contract but a comprehensive and proper project management
plan needs to be initialed as soon as possible. Kasser (1998) introduces lack of or
poor project plan is a significant risk. Besides, an effective project team is consisting
of well trained project developers. Neither too much experienced developers nor too
much inexperienced developers are both inappropriate (Yeates and Cadle, 1996).
Familiar developing methodologies and environment for developers are preferable
(Yeates and Cadle, 1996). Efficient communication among project managers, project
team, customer managers and system final users is the basic insurance for successful
project management and the final victory of information systems implementation.
Nah et al. (2001) introduces that the project must be formally defined in terms of its
milestones (Holland et al., 1999, quoted by Nah et al., 2001). Timeliness of project
and the forcing of timely decisions should be managed (Rosario, 2000, quoted by
Information Systems Project Risk Checklist Lihong Zhou
- 60 -
Nah et al., 2001). Deadlines should be met to help stay within the schedule and
budget and to maintain credibility (Wee, 2000, quoted by Nah et al., 2001). A project
need a high level executive sponsor who has the power to set goals and legitimize
change, and continually strive to resolve conflicts and manage resistance (Nah et al.,
2001).
7.5 Project Technical Aspect
This part of the checklist focuses on those technical issues which might turn into
significant accidents. These risks identified in the table are not only for technicians.
They also require attentions from top managers. Information Systems Project Risk Checklist Part 4of 5
Project Technical Aspect 1 New or unproven developing methodology 2 Unstable or incompatible software system platform 3 Unfamiliar programming language to the developers 4 Unfamiliar developing environment to the developers 5 New or unproven hardware composition 6 Coding from high level requirement without thoroughly design 7 Lack of adequate technology infrastructure
On the technical aspect, a detailed and fully considered technology infrastructure is
the top priority. Both stability and compatibility of hardware and software platform
are highly required. Any instability and incompatibility can bring catastrophic system
crash. Unproven or unfamiliar programming languages, development tools and
methods can lead to project delay, design flaws and project failure. Normally, an
information systems project consists of several similar iterations. In this case,
reusable programming modules can bring a lot of merits to the project. As the reason
of reusable programming codes are not only familiar by programmers, but also
considered as proven and fully tested.
Information Systems Project Risk Checklist Lihong Zhou
- 61 -
7.6 System Development
The final part of checklist looks into specific potential risks involved in each steps of
system development.
Information Systems Project Risk Checklist Part 5of 5
Project Development Development Stages
ID Risk Factors
1 Incomplete comprehension of the current system and current situation of the organization
2 Partially understand the user requirement
System Analyzing and Requirement Defining
3 User requirement are formed only by management level without final system users
4 Inadequate initial general planning before detailed and complex designing
5 Too accurate designing causes the system extremely heavy and unhealthy
6 Without an agreed prototype by the customer
System Designing
7 Poor communication and understanding among the system analyst, designer and the programmer
8 Attempting produce a complete and perfect system in one stage9 Initiate system programming when the system designing is not
completed 10 Accept testing without final users involvement 11 Programming without a preselected and unified methodology 12 Programming without necessary documentations 13 Lack of considering reuseable components from the current
system
System Developing and Acceptance Testing
14 Programmer-oriented system designing instead of user-oriented15 Lack of new system cutover methodologies 16 Insufficient user training
System Training
17 Initiate training without complete testing 18 Lack of a thorough maintenance plan System
Maintaining 19 Assign unqualified staff for system maintenance
The first step for an information systems project is System Analyzing and
Requirement Designing. Preliminary investigation about organization background
and organizational cultural is compulsory. As stated in Chapter 4.2, University of
Sheffield (2000) claims that customers usually miscomprehend between “what we
need” and “What we want”. The investigation also provides a good opportunity to
Information Systems Project Risk Checklist Lihong Zhou
- 62 -
project team to completely understand the user requirement by observation, interview
key people and randomly talk to final system users.
System designing, as the next step, is carried out on the basis of well comprehension
of organization background and project definitions (objectives, scope and
requirement specifications). Drori (1997) advises that the phase requires constant
communicating within the project team; and also between the project team and the
customer. Too detailed designing or too general designing can cause system illnesses.
In order to ensure the project team is producing what the customer want, it is a good
strategy to have customer’s agreement on a prototype system.
A complete system design is the foundation of system developing. The system
developing also needs a minute step-by-step plan. The project team should avoid
producing the system in one phase (Drori, 1997). Programming strategy is defined at
the beginning of the stage. System testing and system training are equally important.
Both of them are required detailed planning before deployment. Moreover, Drori
(1997) indicates that system programming should be initiated after the system
designing is finished; and the final users training should be began after the final
system is fully tested. After the information system development, the organization
needs to initiate a constant system maintenance plan and allocate qualified staff to
implement it.
To conclude, at the beginning of this chapter the concept of how this version of
Information Systems Project Risk Checklist is produced and how to improve this
checklist in the following chapters is explained. The major part of this chapter
dedicates to present and discuss the initial version of Information Systems Project
Risk Checklist. In next 2 chapters, the initial version of checklist is significantly
revised by empirical study of 2 case studies.
Integrated National Crime Information System (INCIS) Lihong Zhou
- 63 -
Chapter 8
Case Study 1: Integrated National Crime
Information System (INCIS)
8.1 Case Background Introduction
The project of Integrated National Crime Information System (INCIS) was designed
to the New Zealand Police with the purpose of supporting operational Policing by
providing improved information, investigation and analysis capabilities. This will
minimize the incidence and effects of crime on the community through the detection
and apprehension of offenders and by crime prevention strategies. The strategic goals
of INCIS are described as:
“The overall strategic goal of ‘reducing the incidence and effects of crime’
transcends traditional crime prevention and seeks to stem and reverse crime trends.
The basic requirements to achieve this ambitious goal are significantly changes in
policies, processes and attitudes within the New Zealand Police.” (Small, 2000:23)
The Executive Summary of Ministerial Inquiry defines the INCIS project is a high
risk project, however it could be achieved with conventional and proved technology
with appropriate time and management, sufficient resources, skills and experience.
The result of INCIS project is achieved some of its objectives but many of its main
objectives are unachievable. IBM, the project partner, didn’t effectively assess and
manage risk, subsequently unable to deliver the system within the time constraint.
Integrated National Crime Information System (INCIS) Lihong Zhou
- 64 -
8.2 Problem Domains Description
8.2.1 Pre-project
1. Changes in Project Scope, Objectives and Requirements
The concept and vision of INCIS project are considerably relevant to the police’s
strategies, but project scope and primary objective of the project are ill defined, and
which significantly changed for several times during the project development.
Modifications of project definitions, such scope and objectives, are considered
catastrophic.
The main concern of the project modified from police strategy to financial objectives
and finally to a technology project. These changes dramatically affected already
defined and approved project scope and requirements, further revisions are required
in this circumstance. Originally, the project plan was to deliver INCIS by Release
One and Two. Under the description of INCIS Liaison Update Newsletter January
1995, Release One was required to install PC networks in all police stations, replace
the legacy with new Suspect and offence Information System, Crime Trend Analysis,
Intelligence Analysis and Mail Facilities by 1997. Release 2 was to implement Case
and Investigation System early 1997. However, the final project was implemented by
3 steps of Increment, which was regarded more focus on software development.
Increment one provided police a system with basic functions, such as electronically
input crime details into an independent database. The core of INCIS system was
Increment two, which significantly improve the basic system from Increment One
into an automatic information system. The implementation of increment two was
intended completely replace the legacy system. Increment Three was planed to add
extra features onto increment two.
The original scope and requirements of INCIS were largely changed as the reason of
implementing Business Process Re-engineering (BPR) after the contract. Although
Integrated National Crime Information System (INCIS) Lihong Zhou
- 65 -
police considered BPR after contract is not harmful, but approximately 8 months of
project delay is indirectly resulted by BPR changes. The Small (2000) emphasizes
the “… the general BPR work needed to be integrated with the INCIS BPR work…. If
BPR had been completed or substantially completed prior to contract, a number of
problems would not have arisen.” (Small, 2000:46)
“…if the BPR is not done first, that the technology will drive the business
requirements rather than the other way around.” (Small, 2000:44)
Risk Identification:
1. Project scope and objectives are ill defined.
2. Significantly change project scope and object after contract.
3. Inappropriate business processes, and reforming them after the contract. 2. Contract
Yeates and Cadle (1996) claim Contract is the major risk in the stage of Pre-project.
All involved parties will be penalized by unsuitable contracting by serious
consequences. The contract for INCIS was signed on 23 September 1994 and with
various obvious pitfalls.
The first contract problem was exposed during Business Process Re-engineering
which enlarged the scope of INCIS. The increment was defined after contract and not
covered by the contract.
“BPR should have been completed or substantially completed before Contract and
before completion of the design of application.” (Small, 2000:44)
“…if BPR had been completed or substantially completed prior to contract, a
number of problems would not have arisen.” (Small, 2000:44)
Integrated National Crime Information System (INCIS) Lihong Zhou
- 66 -
The contract, signed by both parties of police and IBM, was under the circumstance
of a number of unresolved issues in relation to technology, architecture, governance,
management and risks that required to be urgently reviewed by both parties and
INCIS project team. The contract was considered with too many outstanding issues.
The scope and the technology of the project were not satisfactorily defined. The
contract was a fixed or capped priced contract in Release one and an inductive priced
contract in relation to Release two.
“Neither of the parties was ready to enter into contract or to deliver on time or on
budget.” (Small, 2000:151) In the first place, they lack of thorough understanding of
organization status. This factor seriously affected correctly defining project scope
and addressing user requirements. University of Sheffield (2001) suggests that
incorrect project scope and creeping user requirement lead to either building a wrong
system or developing a correct system with bad quality. Moreover, risk management
plan and main risk issues were not specifically addressed before the contract. Small
(2000) points out the contract should not be signed without well-understood, at an
acceptable level, capable of management and accord with detailed risk identification
under the current organization form. A fixed and capped price contract is also
inappropriate. Yeates and Cadle (1996) advise that payment should clearly schedule
and closely connected with sound project milestones and deliverables. A similar
statement was also provided by Small (2000), which claims the schedule of payment
should in accordance to incremental delivery of system modules. Finally, technical
issues and development methodologies were not clarified. This factor induced major
changes with significant impact to the system design and implementation.
Methodologies and technologies substitutions are specifically discuss in 7.2.4 Project
Technical Aspect.
“The Chief Executive should not proceed to contract, unless the risks, as negotiated
in the proposed contract, are well understood, at an acceptable level, capable of
management and in accordance with the risks identified in a business case. Moreover,
Integrated National Crime Information System (INCIS) Lihong Zhou
- 67 -
the greater uncertainty, the more the contractor will increase the price to allow for it.
But there is a second and very major risk; that either party does not understand the
risk uncertainty. If this is the case, then the contract may prove untenable and could
lead to termination.” (Small, 2000:120)
The contract included a backup plan, called Off-Ramp, which was a significant factor
in police side of risk management. The Off-Ramp enabled police to stop the contract
in particular terms and conditions. Police nearly gave up the right of Off-Ramp. The
contract limited that the Off-Ramp could be conducted during a period of 90 days
from a specified commencing date, which is changed for several times. Small (2000)
suggests that “police should have used Off-Ramp in that specific circumstance, or at
least attempted to negotiate a Lay-by.” (Small, 2000:47)
On December 1997, police and IBM initialed a Deed of Variation which specified the
requirements to be achieved by IBM and add traffic, firearms and the justice
interfaces to the project. The cost for the development work to satisfy the
requirement was assessed $20 million which subsequently was proven inadequate.
“The Deed of Variation was considered robust by industry standard but also presents
management challenges to police (Small, 2000:53)”, opinioned by Phillip Fox,
solicitors, on the request of Treasury and Police.
Mr. Carr suggests that “the Contract contained agreement in word but that there was
not agreement in mind and heart.” (Small, 2000:149)
Risk Identification:
1. Lack measurement system for assessing and controlling project risk before
contract.
2. Project scope and objects are ill-defined in the contract.
3. Changing user requirements.
Integrated National Crime Information System (INCIS) Lihong Zhou
- 68 -
4. Unclear payment schedule without linked milestones and deliverables.
5. Lack of, or ignore, backup plans for delay or under performance.
6. Lack of thoroughly comprehension on organizational nature before contract.
7. Technical framework and development methodologies are not firmly defined in the
contract.
8.2.2 Customer The original INCIS plan dated back to 1985. Police had done considerable work to
develop a long term INCIS vision and concept. The entire INCIS project was fully
supported by the New Zealand government and the police. Top management was
quite committed and cooperated to the project.
However, police without an adequate understanding and detailed plan, particularly
the relations among INCIS, current organization form and long term business
strategy. Police significantly change project scope and requirements for several times.
These factors enormously impacted the successful INCIS system delivering.
Additionally, the communication within police is defective. A number of major risks
were possible to be avoided if formal reports were placed or fully considered.
Firstly, Ernst & Young (Australia) was assigned both by police and Treasury to report
pertinent suggestions particularly on strategic considerations, the business case and
the project tender bid. In this report, Ernst & Young mentioned that the report was
remarkably accurate and prescient document, but it was prepared within two weeks;
more importantly, Ernst & Young stated that report is not reliable for
recommendation to proceed with INCIS or to enter into contract, and it is incapable
of playing any important role. Moreover the Executive summary misrepresented the
whole report. However, police generated a paper to recommend Cabinet the
development and implementation of INCIS based on Ernst & Young report.
Integrated National Crime Information System (INCIS) Lihong Zhou
- 69 -
Secondly, in February 1994, a presentation for Minister for Information Technology
from a senior architect of Microsoft stated that the major aspects of the technology
were not deliverable and project is suggested should not be proceeded in that form at
that time. However, police asserted that these problems could be resolved and
recommend the project proceed.
Thirdly, in mid-1994, the Sapphire report identified a number of main business and
technical risks and advised that in order to make sure successfully deliver the INCIS
project these risks need to be resolved and managed. However, police consider these
risks were noticed in the report are those can be managed with a long time strategy
through the life of the project.
Finally, the Deed of Variation is the last remedy of the project. The amendment of
contract required extra cost about $20 million, but the Deed of Variation is signed
without authority from the Cabinet for the increased expenditure.
A number of additional critical issues were summarized by Mr. Steward Watson, who
was appointed as INCIS Project Director in October 1998. “(1) The project,
especially in the police side of the project, had no clear plan that was understood by
all; (2) police had no implementation plan, no training and no strategies in place
when IBM was ready to deliver Increment One.” (Small, 2000:55)
Risk Identification:
1.Changing project requirement, scope and objectives after contract.
2.Final users are unfamiliar with the technology and lack of training.
3.Incorrect decisions made by the project team due to the inefficient communications
within the customer organizations.
Integrated National Crime Information System (INCIS) Lihong Zhou
- 70 -
8.2.3 Project Management
On the project management point of view, the INCIS project management was
notably flawed and didn’t effectively direct the high risk project. The problems of
project management of INCIS project can be classified as follows:
In the first place, project schedule and budget were inadequately defined before
contract, project estimation was poorly generated. University of Sheffield (2001)
suggests that project cost estimation, and quality control are built on the project
schedule plan. The INCIS project experienced several times of changing scope and
requirements, it is reasonable that project management plans must be delicately
revised after changing. The capped priced contract wasn’t made by thorough
deliberation on the detailed project schedule and associated expenditures. As the
reason of unclear defining project schedule and related budgets, project quality
assurance and risk management inevitably can be regarded as on a wrong direction or,
at least, inappropriate.
Besides, as a major part of project management, risk management in INCIS was
significantly insufficient. In the conclusion of Small (2000), “there was little risk
management applied to the INCIS Project until the later stages. Uncertainty was
neither understood nor addressed, and many of the changes and decisions made
during the course of the project added substantially to the risk. Lack of risk
management contributed substantially to the project’s difficulties” (Small, 2000:117)
It is reasonable to appoint a risk management specialist as a risk manager, especially
for the project like INCIS with high risks and high complexity. “Because of the high
risk involved in the INCIS Project, it would have been generally accepted practice to
appoint a Risk Manager, whose rile would have been to ensure the appropriateness
and integrity of the risk management regime.” (Small, 2000:117) The contract is a
tool for transferring defined risks between parties. Thus, it is vital to produce a
complete project risk management plan with specific risk identifications before the
Integrated National Crime Information System (INCIS) Lihong Zhou
- 71 -
contract. “A fixed price contract for the whole of the work of INCIS was
inappropriate and, in fact, increased the risk.” (Small, 2000:71)
Additionally, it is project managers’ obligation to supervise and endorse the project
team to select a fit system developing methodology and technology for the final
information system. The developing methodology is not only required to be
conventional, but also need to be familiar to system developers. A specific technical
infrastructure had been chosen and recorded in the contract including Object
Oriented Technology (OOT), decision support technologies, portability, a process
manager and client/server architecture. Small (2000) states that “although the project
was of high risk, in 1993-1994 the concept or vision of INCIS was sound and should
have been capable of being achieved through the use of conventional technology.”
(Small, 2000:30) However, those technologies that INCIS used were “mostly
unproven for a large complex application” (Small, 2000:70). But in police’s report,
they claimed they had chosen proven leading technology.
The human resource is another project management problem for INCIS. University
of Sheffield (2001) introduces that whilst implementing computerized information
systems, besides managing time and resources, project managers must also many
project personnel, who are selected due to their competency and compatibility.
“Inappropriate staffing, personnel shortfalls” is on Sumner (2000)’s major list of risk
factors in information systems development. During the period of INCIS
development, the project had been managed by 3 project managers in total.
Noticeably, Tony Crewdson, appointed as the project manager in 1994, had extensive
IT experience but he didn’t have experience in managing large IT projects. “The
appointment process needs to ensure the appointment of persons with skill and
experience in the management of large IT projects.” (Small, 2000:44) After the
reassignment of Tony Crewdson in June 1998, the position of project manager was
empty until October 1998. The police chose an emerging developing methodology
but project developers were lack of experience. Yeates and Cadle (1996) advise that
Integrated National Crime Information System (INCIS) Lihong Zhou
- 72 -
project members should be wisely selected according to the project plan and project
objectives, not too many experienced members (cost overruns) and not too many
inexperienced members (effort overruns).
Risk Identification:
1. Unrealistic schedules and budgets.
2. Lack of an effective methodology and poor estimation.
3. Lack of effective risk identifications and risk management plan before contract.
4. Inadequate managing risks after contract.
5. Inexperienced team members in business or technology.
8.2.4 Project Technical Aspect The INCIS project was unique at the time with high complexity and high risks.
“There was no similar project or software package anywhere in the world.” (Small,
2000:30) This factor indicated that INCIS would have to build as a world-first
system by using emerging technology in an unproven way (Small, 2000). Small
(2000) shows that the INCIS objectives were capable are achieved by employing a
conventional technology. Police proposed very specific technical requirements:
Object Oriented Technology (OOT), decision support technologies, portability and a
process manage, and these technologies are on the basis of distributed Object
Oriented Two Tier Client/Server architecture.
However, Small (2000) claims “the technology were mostly unproven and
inappropriate for INCIS project”. (Small, 2000:70) These technologies included:
1. Object Oriented Technology for application, design and development. It is
undeniable that OOT has several significant advantages, however the decision to use
OOT is considered was high risk due to OOT was a very new technology and
standards were immature. There is no previous project in the size and complexity of
INCIS used OOT, particularly in the distributed Object Oriented Two Tier
Integrated National Crime Information System (INCIS) Lihong Zhou
- 73 -
Client/Server architecture. “The decision to use OOT made the implementation of the
INCIS application high risk because it was very new technology and standards were
immature. The methodologies and support tools for designing and developing
business systems using and OOT were not readily available nor was there a pool of
experienced developers.” (Small, 2000:71)
2. The decision of using distributed OO two tier client/sever was brutal. The
technology is very new and without any prove that the technology could meet project
objectives. “In the 1994 to 1997 period, this was an emerging technology and
subsequently has not fully met the claims originally made for it. There is no
successful examples of distributed OO two tier client/server architecture of the scale
(3000 plus users) and geographical distribution when proposed initially in the INCIS
project.” (Small, 2000:72)
3. The process manager was a core INCIS software component which integrating all
other software module together. The process manager was also a new product.
Sapphire report (Small, 2000:279) illustrated that “IBM had no demonstration that
they had the capacity to build the application”. Moreover, the process manager was
planned to be delivered in Release two, but whilst the project implementation the
application was required in Release one. This significantly increased the risk. Finally,
the process manager was substituted by a new software named Flowmark.
4. The decision support system is one of the major objectives that project team was
required to achieve. However, this objective was “very ambitious at that time”
(Small, 2000:73). Police intended to build this system and distribute is to most police
divisions and facilitate most staffs. This consequently required large workload of
local customization and extensive specialized training for police. Finally, the system
was ceased as the contract termination. (Small, 2000)
Integrated National Crime Information System (INCIS) Lihong Zhou
- 74 -
5. Portability also stopped when the contract was terminated. Small (2000) argues
that the application was unable to be finished in the timeframe and the requirement
for portability across a number of operating system platforms.
Generally, infrastructure and application development are the 2 main part of INCIS
technical solution. The technical infrastructure was mainly required to support the
INCIS software applications. The technical infrastructure included computer
hardware (server), system software, System Management Center (SMC), Wide Area
Network (WAN), Local Area Networks (LANs), and computer desktops. The core
architecture for INCIS, represents those technologies used for design the INCIS
system technical architecture, is consisted of Host Operating System MVS, Station
Server Operating System (OS/2 & LAN Server), Desktop Operating System (OS/2 &
LAN Requestor), Host Database (DB2), Station Cabling (Token Ring) and Network
Protocol (SNA). However, police had no technology substitution in spite of “IBM
warned them before contract that it was impossible to achieve the technology
specified”(Small, 2000:71). Besides, developing technologies had remarkably been
changed through system development. These technical substitutions are: Wide Area
Network (WAN) was changed from IBM’s System Network Architecture (SNA) to
Transmission Contract Protocol/Internet Protocol (TCP/IP); Local Area Network
(LAN) was changed from Token Ring protocol to Ethernet; the operating system
OS/2 was substituted by Windows NT; system architecture was changed from
distributed Object Oriented two tier client/server to a three tier client/server
architecture.
Risk Identification:
1. Emerging or unproven, and inappropriate developing technologies and
methodologies.
2. Unfamiliar developing technologies or methodologies for system developers.
3. New or unproven hardware solutions.
4. Significantly change technical infrastructure after system design.
Integrated National Crime Information System (INCIS) Lihong Zhou
- 75 -
8.2.5 Project Development System Analyzing and Requirement Defining: Project scope and user requirements
were not sufficiently defined before contract. Both parties of police and IBM brutally
proceed into contract without selecting proper proven technology specification and
complete project plan estimations. Small (2000) notifies another factor that there was
no proof of concept which is designated to test the proposed approach on a limited
scale. Poof of concept tests the concept, size, and likelihood that the system will
implemented as expected. “There was no comprehensive proof of concept early in
the Project and this was a significant factor in the Project failing to fully achieve its
objectives.” (Small, 2000:139)
System Designing: As the reason of project scope and user requirements were
incorrectly defined before actual project implementation, significant scope and
requirement changes were made after contract. This required immediate project plan
changing. Project technical infrastructures and development methodologies were also
largely modified in the stage of system designing. In this case, project developer
were requested quick reaction in new developing platform and rapid understanding
on new user requirements.
Risk Identification:
1. Emerging or unproven, and inappropriate developing technologies and
methodologies.
2. Project scope and objectives are ill-defined in the contract.
3. Significantly change project requirement, scope and objectives after contract.
Integrated National Crime Information System (INCIS) Lihong Zhou
- 76 -
8.3 Conclusion
INCIS project was incomplete rather than it was a total failure. Small (2000)
illustrates that “INCIS did deliver a number of benefits to police. The project
delivered a technical infrastructure, including approximately 3,000 desktops. The
project also delivered certain aspects of the INCIS application (Increment One)
which was essentially a National Intelligence System (NIS) replacement.” (Small,
2000:57) Nevertheless, the project did not achieve its original objectives. The project
failed on both technical and management sense. The INCIS was a world-first
application using ambitious and emerging technology with great project size and
complexity. However, “although the project was of high risk, in 1993-94 the concept
or vision of INCIS was sound and should have been capable of being achieved.”
(Small, 2000:30) This chapter analysis INCIS project from 5 different perspectives in
association with 5 components of Information System Project Risk Checklist.
London Ambulance Service (LAS) Lihong Zhou
- 77 -
Chapter 9
Case Study 2: London Ambulance Service
9.1 Case Background Introduction
LAS, was started since 1930, generally consists of an Accident and Emergency
Service (A&E) and a non-emergency Patient Transport Service (PTS).
Geographically, LAS is the largest ambulance service all around the world covers
600 square miles and a resident population of over 6.8 million. LAS accept over
5000 patients daily, receives 2000 to 2500 calls everyday.
The system was considered as the first emergency service system all around the
world, especially in the system size and complexity. The LAS planed to use
computerized system completely replace the manual system. The system design was
intended to go as far as possible. The proposed system was an entirely automated
system. In this system most of calls to Central Ambulance Control could be
automatically assigned the most suitable ambulance. Human staffs were only
working on the most complex cases by manually identifying location and allocate the
best resource. It is obvious that LAS management would be largely optimized with
greater efficiency to the command and control emergency service processes.
If the INCIS project was a information system technical failure, then the London
Ambulance Service (LAS) system was a tragedy. The LAS information systems
collapse caused significant delays in ambulances reaching patients on 26 and 27
October 1992 may lead to 20 deaths. Randell (1992) shows the LAS breakdown was
the main reason of people’s death. Finkelstein (1993) concludes that LAS is not a
direct and a single reason of patients’ death based on the 26 cases considered by
coroners’ court.
London Ambulance Service (LAS) Lihong Zhou
- 78 -
The system was not fully tested to a satisfactory level of quality and resilience before
full implementation on 26 October 1992. The following 2 days, 26 and 27 October
1992, were not exceptionally busy days with large amount of additional emergent
incidents and patients. However, the increase incoming calls are unidentified
duplicated calls because of the ambulance delays. “The system failure was not fail in
a technical sense, since generally the system did what is had been designed to do.
However, there were certain fatal system design deficiencies which cumulatively
caused system breakdown.” (Finkelstein, 1993:5) According to the Inquiry Team’s
investigation from the report of Finkelstein (1993), “neither the Computer Aided
Despatch (CAD) system itself, nor the system users were ready for full
implementation on 26 October 1992. The CAD system was not complete, not
properly tuned and not fully tested. The resilience of the system hardware under a full
load had not been tested. The fall back option to the second file server had certainly
not been tested. There were outstanding problems with data transmission to and from
the mobile data terminals. Staff, both in Central Ambulance Control (CAC) and
ambulance personnel, had no confidence on the system and they were not well
trained on using the system. ” (Finkelstein, 1993:35) Moreover, the system users
were working in unfamiliar positions, with no paper backup. The system was unable
to provide precise and complete information.
9.2 Problem Domains Description
9.2.1 Pre-Project Unlike INCIS, pre-project work had fairly well done in LAS which could be
contributed from the first attempt of implementing information systems. LAS had 2
attempts of building computer aided dispatch systems. The first attempt started in the
early 1980s with the main aim was to supply a system without mobile data, but
including a new radio infrastructure. The project was aborted in the 1990 after load
test performance. However, the first project provided LAS a clear definition of what
London Ambulance Service (LAS) Lihong Zhou
- 79 -
kind of system did they want and accurately produce appropriate project scope,
objectives and user requirements.
However, the user requirement producing didn't involve the ambulance crews, which
are a large proportion of final system user. A team was assembled in order to prepare
the requirement specification for the new system. The team was led by the Director
of Support Service, and composed of a System Manager, a Contract Analyst and the
Control Room Service Manager. As the reason of problems at the time with the staff
consultation process, only union representatives were involved in the producing
requirement specification.
Risk Identification:
1. User requirement are formed only by management level without final system users.
9.2.2 Customer “LAS management had received little or no effective management training over the
years.” (Finkelstein, 1993:6) Therefore, it is believable that the management level
staff didn’t have sufficient IT knowledge related to the forth coming LAS project.
Finkelstein (1993) suggests that “new management structures will need to be
accompanied by cohesive, progressive and properly resourced management training
for all tiers of management.” (Finkelstein, 1993:52)
“Poor communications between staff and staff association and senior LAS managers
have created an atmosphere of mistrust.” (Finkelstein, 1993:52) Besides, there was
incomplete “ownership” of the LAS system by the majority of its users. Many
identified problems within the system implementation accelerated the distrust.
Before the project, LAS experienced major culture changes and management reforms.
However, “the result of the initiatives undertaken by management from 1990-92 did
not revitalize management and staff as intended, but actually worsened when was
London Ambulance Service (LAS) Lihong Zhou
- 80 -
already a climate of mistrust and obstructiveness.” (Finkelstein, 1993:3) Some staff
expected the project to fail rather than it to succeed. The proposed new system would
impact very significantly on the way in which staff carried on their jobs. Certain
resistance also appeared on the installation of Datatrack equipment in the ambulance
vehicles. The new system would change the operational method of how ambulance
crews used to work. It was greatly important to ensure ambulance crews’ full
collaborating. There were evidences that crews deliberately damage or misuse the
Datatrack equipment. The impact of this risk is outstanding, and it is greatly
important for managers to overcome this problem. Finkelstein (1993) claims “the
size of the programme and the speed and depth of change were simply too aggressive
for the circumstances. Management clearly underestimated the difficulties involved
in changing the deeply ingrained culture of LAS and misjudged the industrial
relations climate so that staff were alienated to the changes rather than brought on
board”. (Finkelstein, 1993:3) Hirschheim and Klein (1989) provide several
approaches in dealing with user resistance. These approaches rely on a series of
games and strategies make users eventually accept changes. Finkelstein (1993)
advices management must be willing to have regular and open consultation with staff
representatives, and persuade them to overcome their concerns on previous
management approaches, comprehend the need of change and finally accept new
ideas.
It is also questionable in selecting suppliers. In the first place, the selection is lack of
certain criteria. After advertisement, there were 35 companies expressed an interest
in providing all or part of the system. LAS generated a checklist protocol in selecting
appropriate suppliers. Each prospective supplier was evaluated and scored on a
consistent to the checklist. In order of importance, these weighting factors were:
ability to perform the tasks required; ability to handle throughput and response time;
ease of use by staff; resilience; flexibility; ability to meet time table; cost; additional
feature. However, there are evidences that LAS didn’t follow up the checklist to
choose the highest ranking company, but followed the instructions that choose the
London Ambulance Service (LAS) Lihong Zhou
- 81 -
supplier with the lowest price. Finally, the SO was the only company can met the
requirements from LAS, and had been selected as the prime supplier with the
optimum solution of a system consisting of Apricot, System Options (SO), and
Datatrak. In the beginning, SO was not optimistic about biding for this contract and
they were unsuccessful in biding for more basic system for the Combridgeshire
Ambulance Service. The SO price for developing LAS system was amazing cheaper
than other bidders, which was skeptical and questioned by many bidders. SO is a
well established, small software company with a good reputation amongst their many
prior customers in providing satisfied system technical quality. Finkelstein (1993)
indicates that “within the time constraints imposed on the project and the scope of
the requirement, no software house could have delivered a workable solution. ……
In taking on the LAS project, which was far larger than anything they had previously
handled, so rapidly found themselves in a situation where they became out of their
depth.” (Finkelstein, 1993:21)
External society gave constant pressure on LAS to improve performance. LAS
management expected the implementation of information systems can upgrade LAS
service level and release the pressure. On the one hand certain amount of pressures
can ensure the project be finished in time and with good quality, on the other hand
too much pressures can force the project team setup rigid project deadlines and select
unproven technologies with ambitious project objectives. “….faced with concerted
pressure from its managing RHA, MPs, the public, health service consumers and the
media over improving performance times, it is by no means certain that the Service
would have been allowed to adopt a more measured approach to introducing
changes, particularly with CAD.” (Finkelstein, 1993:6)
Risk Identification:
1. Management in the customer had received over the years little or no effective
management training.
2. Management doesn’t understand technical issues.
London Ambulance Service (LAS) Lihong Zhou
- 82 -
3. Mistrust between management and staff.
4. Lack of confidence on the project.
5. Inappropriately select the system supplier with ambiguous selecting criteria.
6. The supplier is lack of experience or unsuitable for the particular project.
7. Constant external pressures and unable to properly manage them.
9.2.3 Project Management
The LAS project management was quite inadequate. It was flawed in following ways:
1. The ambitious LAS project was built on an impossible time schedule and budget.
The project group meeting on 17 June 1991 identified the improper timetable, and
the meeting minutes states that the industry average project duration in terms of
similar project size and complexity as LAS was approximately 18 month, but the
LAS system was asked fully implemented by January 1992 which was no more than
6 months and non-negotiateable. The timetable didn’t allow any revision and rework
on the system, everything must be coming right in the first time. LAS management
and LAS project team had a proposed budget in mind, for the whole system, of
around £1,500,000. The time schedule and budget were fixed and unchallengeable.
They were not made by carefully estimations and thoroughly calculation. Finkelstein
(1993) warns that “it was dangerous of setting unrealistic timescale without
consultation, and commitment of, those involved; and the public and its
representatives must be prepared to allow the LAS breathing space to put its house in
order.” (Finkelstein, 1993:6)
2. LAS project team chose PRINCE Project Management Methodology. However,
neither LAS nor the suppliers had direct experience in using the methodology.
Although a course was carried on for main project members to learn how to
executive the PRINCE, it is clear that the project team didn’t have actual real project
management experience in using PRINCE. “Although certain elements of the
London Ambulance Service (LAS) Lihong Zhou
- 83 -
PRINCE methodology were used, at least in the initial stages, it was not used in a
properly structured way through the duration of the project.” (Finkelstein, 1993:21)
3. It is stated that the LAS system was the “World First” pioneering new system
developed by leading technologies and methodologies. It was unacceptable that there
was no a full-time committed project management team. The project team was lead
by the Director of Support Service, and involves LAS Contract Analyst, Control
Room Service Manager, Director of Operations, Training Manager, Public Affairs
Manager, CTS representative, Administrative Support, Supplier representatives. But
none of them are fully committed to the project management. Finkelstein (1993)
suggests “professional project management might have resolved disputes between
suppliers more quickly and would have identified the fact that suppliers were often
providing optimistic reports of their progress. It is probable also that a professional
project manager would have questioned and “go behind” some of the cosy
assurances on the project progress that were being given by the suppliers.”
(Finkelstein, 1993:22)
4. In LAS project, quality management largely was supervised by Project Issue
Report (PIR). All quality related problems with software or any other aspect of the
system were communicated to the suppliers by the PIR. The PIR was designed to
monitor quality control and identified quality problems. Generally, the PIR report
system was a fairly efficient way to ensure project quality. However, PIR was not a
complete quality control plan which only identified negative quality issues but
without certain solutions to associate with them. Besides, “certain procedures were
often circumvented by SO who would on occasions amend software to meet
individual user’s wishes without going through the PIR route.”(Finkelstein, 1993:33)
By the 26 October 1992, the total number, of PIR submitted development problems
and software quality issues, was 1,513. 81 of them were remaining outstanding on
that date. Among the 81 issues, 2 were considered would not function in the
operational environment until this is rectified. 44 were considered would degrade the
London Ambulance Service (LAS) Lihong Zhou
- 84 -
quality of service to patient. LAS required a more efficient way to manage project
quality.
5. Lack of a complete risk management strategy in LAS project. Based on University
of Sheffield (1996) risk management is extremely important to information systems
projects. Risk management provides a disciplined environment for proactive decision
making to: assess continuously what could go wrong (risks); determine which risks
are important to deal with; implement strategies to deal with those risks. In spite of
Senior management, the project team, and the lead supplier had fully committed to
the project and constantly contribute their best efforts, they insufficiently identified
the major risks that were eventually cause the project failure, which mainly as the
result of the no formal risk management.
6. Project management was difficult to be implemented due to the ambiguous
relationships between contractors. On the perspective of LAS, they prefer Apricot as
the lead contractor rather than SO. SO was a very small software house but Apricot,
was entirely owned by Mitsubishi, was a much stronger party. In SO’s mind, Apricot
was also more appropriate as the lead contractor. In this case, SO could take off some
of the pressure off them, particularly in terms of project management. However,
Apricot declared that it is corporate policy to take on such a lead position if it is in a
position to control the project itself. Finally, a decision was made that SO would be
the lead contractor whereas they were incapable to take the position. “…SO would
need to demonstrate their previous experience and their ability to project manage a
project of this size. This concern was not specifically followed up, but LAS gained
implicit confidence from the fact that Apricot themselves must have had that
confidence. In fact Apricot did no positive vetting of SO and their abilities to deliver
on the contract.” (Finkelstein, 1993:30)
Risk Identification:
1. Unrealistic and inflexible project timelines and budget.
London Ambulance Service (LAS) Lihong Zhou
- 85 -
2. Lack of, or choose an unfamiliar project management methodology.
3. Team members are not fully committed to the project.
4. Lack of effective risk management plan.
5. Lack of a complete quality assurance methodology.
6. Ambiguous relationships of contractors, and disagreement between involved
partners after contract.
9.2.4 Project Technical Aspect The Apricot consortium was selected as the final technical solution which included Apricot,
System Options (SO), and Datatrak. Mainly, the entire system was consisted of 3 components:
Computer Aided Despatch (CAD); Computer Map Display (CMD); Automatic Vehicle Location
System (AVLS). This system was designed interface with the existing SOLO Mobile Data
Terminals (MDTs) implemented as a part of the prior aborted system. The complete CAD system
had 7 parts, namely CAD Software, CAD Hardware; RIFS Communication Interface; Radio
System; Datatrak Sub System; GazeKeer and Mapping Software; Mobile Data Terminals.
The hardware systems were very powerful in LAS. The computer hardware is industry standard
“IBM compatible” work stations and file servers. Most of the processing was finished in
workstations and, comparatively, file servers were not playing the key roles. All workstations
were 486 based 25mhz (the most powerful chip in standard use). The workstations were using
Microsoft Windows 3.0 environment which was a powerful operation system in providing an
user friendly interface and simultaneous multi-threading function.
However, there were various fatal technical problems about the new system The
system was fully deployed on the day of kickoff with these 2 serious faults.
1. Occasionally, Mobile Data Terminals provided incorrect information.
2. The system was running in a slow speed of response. There are 3 inseparable
possible reasons according to Finkelstein (1993): “inefficient software design;
insufficiently powerful hardware; operating environment insufficiently powerful to
support the application design.”(Finkelstein, 1993:34) These 3 issues are
London Ambulance Service (LAS) Lihong Zhou
- 86 -
interconnected, but the inefficient software design was the major reason of system
delay. Finkelstein (1993) points out that the project development team
inappropriately selected Visual Basic as the main design tool. But Visual Basic is a
convenience tool for fast system development rather than the development of fast
systems. “SO were using an, at the time, unproven development tool designed
primarily for prototyping and the development of small, non mission critical
systems.” (Finkelstein, 1993:34)
3. Based on Finkelstein (1993)’s study, low response speed was not the only problem
about the Visual Basic. Early system failures might caused by the development
platform of a combination of Visual Basic and Windows 3.0. “Basic routines within
Windows 3.0 have resulted in some unexplained system crashes.” (Finkelstein,
1993:34)
4. The entire system was not fully installed or tested by the day of kickoff according
to Finkelstein (1993).
5. The Datatrak system has been considered to be occasionally unreliable during the
system development and implementation phases. Datatrak is arguably accurate, thus
the investment on Datatrak can be retained. “Work will need to be done on improving
the accuracy and reliability of the system, possibly involving a change to the method
used to calculate location information. This will require LAS to work in close
partnership with Datatrack.” (Finkelstein, 1993:46)
Risk Identification:
1. Emerging or unproven, and inappropriate developing technologies and
methodologies.
2. Systems are not fully tested before final implementation.
3. Unstable or incompatible software and hardware system platform.
4. Constant close partnership between the customer and the supplier.
London Ambulance Service (LAS) Lihong Zhou
- 87 -
9.2.5 Project Development System Analyzing and Requirement Defining:
LAS clearly knew what kind of system they want and what main functions they need.
Project scope and objectives were outstandingly defined prior the actual system
development. Basically, the proposed Computer Aided Despatch (CAD) system
included the primary command and control functions of: call taking (receiving and
verifying detailed incident information; resource identification (decide which
ambulance to send); resource mobilization (allocate the most appropriate ambulance
to the incident); ambulance resource management (managing the ambulance
equipment and crews). Project requirement specifications were also generally well
presented, though there was little involvement from ambulance crews in requirement
generating. The system requirement specification was written right after the
abandonment of the first attempt of CAD system. It is inevitable that the first project
provided an opportunity to thoroughly understand the fundamental organization
requirements and accurately define project scope.
System Designing:
A combination of Visual Basic and Windows 3.0 was chosen as the primary
developing environment. Visual Basic, at that time, was unproven development tool.
The developing platform was also without any confirmations about its precision and
stableness. Evidences proved that the platform of combination caused serious system
delay which was one of major risks of final system collapse (detail information is
available in 9.2.4). Finkelstein (1993) concludes the CAD system and its supporting
infrastructure were flowed in “a need for near perfect input information in an
imperfect world; poor interface between crews, MDTs and the system; unreliability,
slowness and operator interface.” (Finkelstein, 1993:39)
System Developing and Acceptance Testing:
The LAS project team was fully aware the leading edge nature and complexity of the
proposed system. The system was developed on an unstable and unproven platform.
London Ambulance Service (LAS) Lihong Zhou
- 88 -
The system was not completely installed and tested until the day of kickoff on the
26th October 1992. “The completeness and quality of system testing is in doubt,
owing to the eventual piecemeal delivery and implementation of the system. Over the
following months various system elements were tested, but never as a fully integrated
whole” (Finkelstein, 1993:23)
Both functional and quality testing was operated partially finished system in January
1992. At that time various key components were not finished. Although these key
parts were tested in the following month, the complete and integrated system test had
never thoroughly tested. Another problem about the system testing was some random
failures were difficult to be simulated and can only happen in the real time.
Finkelstein (1993) suggests LAS project team built up a consistent automatic testing
program with sufficient test data to deal with such un-manual testable problems.
System Training:
Staff training had unsatisfactorily addressed in LAS. Mainly, there are 2 main critical
risks:
1. Significant “skill decay” between the time of training and actual implementing the
system in use, though the training both for Ambulance Crew and Central Ambulance
Control was carried out well in advance of the planned implementation date. Drori
(1997) indicates that “final users training should be began after the final system is
fully tested”.
2. There were also doubts on the quality of the training delivered both from LAS’s
own work based trainers and OS. System users received different levels of training
depending on what specific part of the system they use. Besides, a key issue of LAS
failure on the CAD system was ambulance staff and CAC didn’t fully appreciate
each other’s role in providing the service to London. This problem was produced by
separate training of these 2 aspects. Certain parts of the system could be trained
jointly to both sides. Finkelstein (1993) claims “This problem was probably
London Ambulance Service (LAS) Lihong Zhou
- 89 -
exacerbated by totally separate training of these two functions. Greater “buy in” to
the system would have been achieved had at least certain elements of the CAD
training been given jointly to both sides. This would have enabled them each to
understand how successful operation of the system could only come about through
CAC and the crews operating in full partnership. It would also have enabled them
each to better understand the stresses that the other works under.” (Finkelstein,
1993:31)
Risk Identification:
1. User requirement are formed only by management level without final system users.
2. Unstable or incompatible software and hardware system platform.
3. Inadequately test the final integrated system before final implementation.
4. Initiate training without complete testing.
5. Lack of a user training plan and insufficient user training.
9.3 Conclusion
INCIS project might just was an outstanding system failure, but LAS was definitely a
tragedy. Not just because of high financial cost of £1.1–£1.5 million, but more
importantly 20 people may have died indirectly caused by system failure. From the
case analysis in this chapter, there were significant risks both on technical and
management aspect. However, Finkelstein (1993) claims in the final project report
“on 26 and 27 October 1992 the computer system itself did not fail in a technical
sense. Response times did on occasions become unacceptable, but overall the system
did what it had been designed to do. However, much of the design had fatal flaws
that would, and did, cumulatively lead to all of the symptoms of system failure.”
(Finkelstein, 1993:5)
London Ambulance Service (LAS) Lihong Zhou
- 90 -
These two case studies have a lot of similar perspectives, such as they were both the
world-first computer system, which can highlight most significant risks. However,
this research pays more attentions on the differentiations in these cases. These
differentiations greatly increase the coverage of the empirical study and provide
opportunities to evaluate the Systems Project Risk Checklist on more distinct facets.
The table below shows similarities and differentiations details of these 2 case studies: Area Topic INCIS LAS
Project Definition √ √ Pre-Project
Contract √
Management Level √ √
Internal Difficulties √
Communication Barrier √ √
Customer
System Supplier Selection
√
Project Management Methodology
√
Poor Estimation √ √ Risk Management and Quality Assurance
√ √
Project Team
Project Management
Human Resource √ √
Developing Technology and Methodology
√
Developing Platform √
Project Technical Aspect
Developing Environment
System Analysis and Requirement Defining
√ √
System Designing √
System Developing and Acceptance Testing
√
System Training √ √
Project Development
System Maintaining
Figure 23: Compare INCIS and LAS
London Ambulance Service (LAS) Lihong Zhou
- 91 -
This table clearly shows similarities and differentiations between INCIS and LAS
case studies. The table is divided into 5 main areas according to the Information
Systems Project Risk Checklist. However, this table is not intended to illustrate
details information as the risk checklist. Therefore these 5 areas are further sub-
divided into distinct topics.
In the Pre-Project area, Project Definition stand for risks of inappropriately define
project scope, objectives and requirement specification before the contract and
significantly change them after system designing. Both INCIS and LAS had risks in
Project Definition. Contract means any risks could relate to project contract. Only
INCIS project involved significant contract related risks. In the Customer area, both
INCIS and LAS management were quite support and committed to these project.
Both INCIS and LAS had serious internal and external communication barriers
which is an outstanding risk. LAS had Internal Difficulties about mistrust and lack of
confidence on the project. The system suppliers for LAS project was also considered
inappropriate. In the aspect of Project Management, apart from LAS selected
unfamiliar project management methodology PRICE, both LAS and INCIS had risks
in poor project estimation, risk management and quality assurance, and human
resource (assign inexperienced project managers). On Project Technical Aspect,
INCIS showed risk in Developing Technology and Methodology; LAS incorrectly
selected developing platform.
LAS presented a large scale of System Developing risks, however INCIS project
showed risks in System Analyzing and Requirement Defining and System Training.
As illustrated in the table, topics of Project Team in Project Management,
Development Environment in Project Technical Aspect and System Maintaining in
System Development are not covered by these 2 cases. However, these topics are
covered and compensated by literature review.
Formal Information Systems Project Risk Checklist Lihong Zhou
- 92 -
Chapter 10
Formal Information Systems Project Risk Checklist
The purpose of this chapter is to revise the initial Information Systems Project Risk
Checklist from the case studies of previous 2 chapters. In the mean time the present
the main research result, the formal Information Systems Project Risk Checklist, of
this dissertation.
New identified risks and modified risks from the case studies are marked with “*”.
10.1 Pre-Project
Information Systems Project Risk Checklist Part 1of 5
Pre-Project 1 Complex and unclear relationship among partners, customers and suppliers 2 Disagreement between contract involved partners 3 Failure to consider existing relationships when replacing systems 4 Failure to consider existing systems when replacing systems 5 Lack of senior management support 6 Project scope, objective and requirement specifications are ill defined 7 Resources are not allocated well 8 The customer lack of formal experience 9 Unclear payment schedule without linked milestones 10 Unsuitable business plan and vision 11 Without, or ignore, backup plans for delay or under-performance * 12 Significantly change project scope, objective and requirement specifications
after contract * 13 Inappropriate business processes, and reform them after the contract * 14 User requirements are formed only by management level without final system
users * 15 Lack measurement system for assessing and controlling project risk before
contract * 16 Lack of thoroughly comprehension on organizational nature before contract * 17 Technical framework and development methodologies are not firmly defined in
contract *
Formal Information Systems Project Risk Checklist Lihong Zhou
- 93 -
The project scope, objectives were experienced significant modifications which were
considered as major causations of INCIS project failure. Business Process Re-
engineering (BPR) after contract required changes in project scope, objectives and
user requirement specifications which indirectly result approximately 8 months of
project delay. In contrast, pre-project work had fairly well done in LAS which could
be contributed from the first attempt of implementing information systems. However,
the LAS’s requirement specifications were formed only by management without final
system users. In INCIS project contract, technical framework and development
methodologies were not firmly defined which could be the reason of major changes
during project development. In INCIS, the Off-Ramp enabled police to stop the
contract on particular terms and conditions, but police nearly gave up the right of
Off-Ramp.
Formal Information Systems Project Risk Checklist Lihong Zhou
- 94 -
10.2 Customer
Information Systems Project Risk Checklist Part 2of 5
Customer 1 Significantly change project scope, objectives and requirement specifications
after contract
2 Conflicts between user departments
3 Final Users are unfamiliar with the technology and require additional training
4 Internal political difficulties
5 Lack of customer support
6 Reluctance of changing environment to accept the new system
7 Resources are not well assigned to the project team
8 Unable of unifying customers perspective on the project
9 Inefficient communication between involved parties *
10 Management in the customer received over the years little or no effective
management training *
11 Management doesn’t understand technical issue *
12 Mistrust between management and staff *
13 Lack of confidence on the project *
14 Inappropriately select the system supplier with ambiguous select criteria *
15 The supplier is lack of experience or unsuitable for the particular project *
16 Constant external pressure and unable to properly manage them *
In INCIS case, various warning signs during the system development had been
proposed, but they didn’t pay enough attentions by police due to lack of efficient
communication within police. In LAS case, inefficient communication caused
several major risks: mistrust between management and staff; lack of confidence to
the project; system supplier optimistically report the system development status to
customer. Additionally, LAS inappropriately selected the system supplier, SO,
without clear selecting criteria. So also had been proven have no sufficient
Formal Information Systems Project Risk Checklist Lihong Zhou
- 95 -
experiences on the project in size and complexity as LAS system. External society
gave constant pressure on LAS to improve performance. LAS management expected
the implementation of information systems can upgrade LAS service level and
release the pressure. However, LAS unsuccessfully managed the pressure. The
unclear “ownership” of the LAS system by the majority of its users worsens the
relationships between departments within LAS. Significant user resistance also
existed in the LAS.
Formal Information Systems Project Risk Checklist Lihong Zhou
- 96 -
10.3 Project Management
Information Systems Project Risk Checklist Part 3of 5
Project Management 1 Lack of an effective methodology and poor estimation 2 Unrealistic and inflexible timeliness and budgets * 3 Lack of effective risk identifications and risk management plan before contract,
inadequate managing risks after contract. * 4 Milestones and related deliverables are ill-defined 5 Inexperienced team members in business or technology 6 Inappropriate structure of project team 7 Team members are not fully committed to the project 8 Too many junior staff or too many senior staff 9 Unfamiliar environment for system developers 10 Lack of process and standard 11 No single person is accountable/responsible for the project 12 Client and development staff fail to attend scheduled meeting 13 Inappropriate staffing, personnel shortfalls 14 Lack of, choose an unfamiliar project management methodology * 15 Lack of a complete quality assurance methodology * 16 Ambiguous relationships and disagreement between involved parties after
contract *
Both of INCIS and LAS project schedule and budget were inadequately defined
before contract, project estimation was poorly planned. INCIS’s project scope and
requirement significantly changed during implementation. The capped priced
contract wasn't made on associated project schedule and budget. LAS made an
unrealistic and inflexible timescale which largely increased the difficulty of the
project. There was little risk management applied to both of INCIS project and LAS
project. Project teams for both INCIS and LAS were inappropriately constructed.
INCIS didn't appoint experienced project managers in the project; LAS had no full-
time committed project management team.
Besides, for INCIS project, developing methodologies and technologies were
inappropriately selected in the first place, and subsequently experienced major
changes during the system implementation. For LAS project, project team selected
an unfamiliar project management methodology which was an outstanding risk; LAS
Formal Information Systems Project Risk Checklist Lihong Zhou
- 97 -
quality assurance control was ineffective and incomplete; unclear contractors’
relationships between contractors made the project management difficult to be
implemented.
Formal Information Systems Project Risk Checklist Lihong Zhou
- 98 -
10.4 Project Technical Aspect
Information Systems Project Risk Checklist Part 4of 5
Project Technical Aspect 1 Emerging or unproven, and inappropriate developing technologies and
methodologies * 2 Unstable or incompatible software and hardware system platform * 3 Unfamiliar developing technologies or methodologies for system developers * 4 Unfamiliar developing environment to the developers 5 Coding from high level requirement without thoroughly design 6 Constant close partnership between the customer and the supplier after the
project * 7 Significantly change technical infrastructure after system design
For INCIS’s technical infrastructure, Object Oriented Technology and Object
Oriented two tier client/server were emerging and unproven. Major requirements on
project functions of process manager, decision support system and portability were
difficult to be implemented with great complexity. It was also an outstanding risk to
change the technical infrastructure after system design. In LAS project, the system
supplier SO was using, at that time, an unproven development tool, a combination
software platform of Visual Basic and Windows 3.0, designed primarily for
prototyping and the development of small, non mission critical system. The final
system was neither fully installed nor tested by the day of system kickoff. The
system was running in a slow speed of response. Additionally, Mobile Data
Terminals provided additional incorrect information.
Formal Information Systems Project Risk Checklist Lihong Zhou
- 99 -
10.5 Project Development
Information Systems Project Risk Checklist Part 5of 5
Project Development Development Stages
ID Risk Factors
1 Incomplete comprehension of the current system and current situation of the organization
2 Emerging or unproven, and inappropriate developing methodologies and technologies *
3 Partially understand the user requirement 4 Significantly change project scope, objectives and requirement
specifications *
System Analyzing and Requirement Defining
5 User requirement are formed only by management level without final system users
6 Inadequate initial general planning before detailed and complex designing
7 Unstable or incompatible software and hardware system platform.*
8 Too accurate designing causes the system extremely heavy and unhealthy
9 Without an agreed prototype by the customer
System Designing
10 Poor communication and understanding among the system analyst, designer and the programmer
11 Attempting produce a complete and perfect system in one stage12 Initiate system programming when the system designing is not
completed 13 Accept testing without final users involvement 14 Inadequately test the final integrated system before final
implementation * 15 Programming without a preselected and unified methodology 16 Programming without necessary documentations 17 Lack of considering reuseable components from the current
system
System Developing and Acceptance Testing
18 Programmer-oriented system designing instead of user-oriented19 Lack of new system cutover methodologies 20 Lack of a user training plan and insufficient user training *
System Training
21 Initiate training without complete testing 22 Lack of a thorough maintenance plan System
Maintaining 23 Assign unqualified staff for system maintenance
In INCIS case, project scope and user requirements were not sufficiently defined
before contract. Both parties of police and IBM brutally proceed into contract
without selecting proper proven technology specification and complete project plan
Formal Information Systems Project Risk Checklist Lihong Zhou
- 100 -
estimations. As the reason of project scope and user requirements were incorrectly
defined before actual project implementation, significant scope and requirement
changes were made after contract. Although in LAS project the project scope,
objectives and requirement specification were well planned out before the contract,
in the system designing the project team chose an unstable developing platform. The
LAS system was tested piece by piece during the system implementation, but the
final integrated system was not fully tested. Both functional and quality testing was
operated partially finished system in January 1992. At that time various key
components were not finished. Staff training was poorly planned and implemented in
the LAS. Significant “skill decay” between the time of training and actual
implementing the system in use. The quality of the train was also doubtful.
Conclusion and Future Research Lihong Zhou
- 101 -
Chapter 11
Conclusion and Future Research
Researches on information systems failure in recent decades haven’t outstandingly
decreased the astonishing rates of failures. This research project also dedicates to
study on possible reasons and potential risks in information system failures.
11.1 Research Objectives
As noticed in the chapter 1 introduction, the main purpose of this research is to build
up a complete checklist, which can identify most common risks underlying in
information system project. The risk checklist could be used as an elementary device
for information systems developers in assessing and avoiding potential risks.
Figure 24: Diagram of revisions on Information Systems Project Risk Checklist
Two versions of information systems project risk checklist has generated from this
study. The initial version of risk checklist is constructed by risk factors identified
from recent publications on critical factors of information system failures, based on a
research foundation built by substantial literature review on information systems
implementation, nature of failure, project management and risk management. The
major part of literature review is contributed to theory and research foundation
building which provide a research platform for subsequent empirical study.
Conclusion and Future Research Lihong Zhou
- 102 -
The formal, second, version is presented after the cases analysis by using the first
version of risk checklist. The purpose of case studies is not just attempt to apply the
checklist to real life cases, but more importantly, to significantly revise the first
version of risk checklist. Cases of Integrated National Crime Information System
Project (INCIS) and London Ambulance Service (LAS) are selected. These two cases
are both well-known information system disasters with large scale of project scopes,
high project costs and high technical complexity which provide great opportunities to
investigate into real information system failures.
The main purpose and major objectives of this research have been successfully
completed. An overview of this project is provided in the subsequent section in this
chapter.
11.2 Research Processes
The entire research thesis can be divided into two major parts as the main
methodologies of how they are studied (Literature review and Case study).
Figure 25: Dissertation Structure Map
Conclusion and Future Research Lihong Zhou
- 103 -
As shown in the picture above, extensive review of literatures, on Information
System Implementation (Chapter 3); Nature of Failure (Chapter 4); Project
Management (Chapter 5) and Risk Management (Chapter 6), consisted the basic
research foundation.
After Chapter 1 Introduction and Chapter 2 Research Methodology which explain the
research intention and how the research is actually conducted, Chapter 3 Information
Systems Implementation introduces basic definitions and the importance of
information system. Nature of failure in Chapter 4 not only shows why study failure,
but also explains advantages of study information system implementation from its
failures. Project management and Risk management are analysis and discuss in
Chapter 5 and 6 respectively.
The Chapter 7 can be considered as the end of literature review, and the start of
empirical study. Chapter 7 presents an initial version of Information System Project
Risk Checklist from literature review. Then, the initial risk checklist significantly
revised in INCIS (Chapter 8) and LAS (Chapter 9) case studies by assessing risks
within these 2 real life environment. At then end of the chapter 9, these 2 cases are
compared. A formal Information System Project Risk Checklist, which is the main
research result of this dissertation, is presented in Chapter 10. This version of risk
checklist is significantly revised from the initial version. Detailed information and
discussions about how this checklist was revised is available in Chapter 10.
11.3 Research Methodology
This dissertation research is conducted by using an inductive methodology where
literature review and 2 cases studies based empirical study have been carried out. As
introduced and discussed in the section 2.3 Research Approach in Chapter 2,
inductive research approach is more appropriate than deductive research approach
Conclusion and Future Research Lihong Zhou
- 104 -
for studying risk factors in information system project. Inductive reasoning starts
from problem statement and then generate conclusion from existing data which
approach uses current data to determine the key concept to be discuss. For this
dissertation, it is advantageous to use inductive reasoning as the primary research
approach.
This dissertation mainly uses secondary data source. The thesis not only employs
advantages of secondary data (efficient, easy access, high quality and permanently
available), but disadvantages are also effectively avoid. Qualitative data analyses
methodology is selected in this research.
Literature review and case study are basic research approaches of this research
project. An initial version Information Systems Project Risk Checklist has been built
by extensive literature reviews. Risks illustrated on the risk checklist were selected
from previous academic research publications. The subsequent cases studies
evaluated the risk checklist with the real life environment and provide further risks
which are not covered in literature review.
11.4 Research Result
As introduced at the beginning of the thesis, the main purpose is to produce a
thorough risk checklist as an elementary tool for information systems project
developers by extensive literature review and case studies.
The research result, the formal Information Systems Project Risk Checklist
(presented in Chapter 10), has 5 main components. “Pre-Project” shows potential
risks that project team needs to beware before the actual system implementation.
Customer fully cooperation is critically important to the project. The second part of
the checklist, “Customer”, points out risks within customers. As introduced in
Chapter 5, project management provides efficient and effective tools to manage
Conclusion and Future Research Lihong Zhou
- 105 -
information projects, assists project team eventually successfully implement
information systems. “Project Management” part of the risk checklist lists possible
critical factors and mistakes could take place on project management aspect. “Project
Technical Aspect” concentrates on project technical issues and “Project
Development” explains project risks phase by phase.
Risks identified by the Information Systems Project Risk Checklist are from
literature review and analysis on INCIS and LAS case studies. These risks cover
nearly every aspect of information systems implementation. Empirical study
conducted in this thesis also proved that the risk checklist can efficiently and
sufficiently identify most of risks in information system projects. The risk checklist
can effectively provide elementary suggestions for information systems developers
and can be used as a risk avoiding tool.
An Information System Project Risk Checklist is also available in the Appendix I.
11.5 Research Contribution and Future Research
“After decades of experience and countless publications, most IT projects still fail to
reach a satisfactory conclusion.” (Hyde, 2002, quoted in Liu, 2002:66)
After nearly half a year of reading on information systems implementation, and three
months of investigation on information system failure, there is a large amount of
researches have been contributed to the area of information systems failure. However,
they are still insufficient to actually solve the problems of information systems
failures. These studies are critisized as incomplete and partial since they mainly
focus on most significant risks of information systems projects failures.
Conclusion and Future Research Lihong Zhou
- 106 -
This research, instead of looking into some specific failures or deeply study several
particular risks, intends to create an Information System Project Risk Checklist. This
list highlights most of those outstanding risks which can possiblely causes
information system failures. The Information System Project Risk Checklist is
intended to be built as an handy and elementary information systems project risk
avoidance device.
The Information Systme Project Risk Checklist can be used:
a) to build project management, risk management and quality assurance strategy;
b) to directly remind system developers about significant potential risks;
c) to objectively assess information systems projects during system implementation;
d) as an effective guidance for project managers and system developers to avoid
outstanding risks or, at least, reduce negative impacts from those risks.
e) as an easy tool for developers and evaluators to evaluator current project status at
any stage and on any facet during system implementaton.
f) as a convinent device for information researchers to evaluate any cases in any
environment.
Futher research can be undertaken based on research result of this dissertation
It is necessary to continuously investigate not only cases of information systems
failures, but also implemented unsatisfactory information systems and even
successfully developments.
Risks need more than just be identified, they also require further studies to point out
approaches to deal with these risk, or in some occasions turn risks into advantages.
The Information Systems Project Risk Checklist needs to be constantly be revised.
More risks should be identified and evaluated in further researches.
Conclusion and Future Research Lihong Zhou
- 107 -
The Information System Project Risk Checklist can be further researched in how to
coorperate with project strategic planning.
The definition of information systems is a broad conception for computer-based
information technologies. Information System Project Risk Checklist can be further
developped for some particular projects, such as information systems for hospitals,
hotels, schools, etc.
Reference Lihong Zhou
- 108 -
Reference
Ackoff, R.L. (1967)."Management misinformation systems". Management Science,14,4,
Dec.
Ackoff, R.L. (1994). "It's a mistake!" Systems Practice, 7, 3-7.
Aiken, P. (2002). "Enterprise Resource Planning (ERP) Considerations."
Argyris, C. (1971). "Management information systems: the challenge to rationality and emotionality". Management Science, 17, 6, Feb (B-275-292). Avison, D. & Fitzgerald, G. (2003). Information Systems Development: Methodologies, Techniques, and Tools. McGRAW-HILL PUBLISHING COMPANY.
Avison, D. & Shah, H. (1997). The Information Systems Development Life Cycle: A
First Course in IS. McGraw Hill.
Awad, E. (1988). Management Information Systems: Concepts, Structures &
Applications. Benjamin-Cummings Publishing Company.
Bandyopadhyay, K., Mykytyn, P,. Mykytyn, K. (1999). "A Framework for
Integrated Risk Management in Information Technology". Management Decision, 37
(5), 437-444.
Beynon-Davies, P. (1999). "Human error and information systems failure: the case
of the London ambulance service computer-aided despatch system project."
Interacting with Computers, 11, 699-720.
Beynon-Davies, P. (2002). Information System: An Introduction to Informatics in
Organizations. Palgrave.
Reference Lihong Zhou
- 109 -
Bignell, V. & Fortune, J. (1984). Understanding Systems Failures. Manchester:
Manchester University Press.
Boddy, D., Boonstra,A.& Kennedy, G. (2005). Managing Information System: An
Organisational Perspective. Second Edition. . Prentice Hall Financial Times.
Boehm, B. & DeMarco, T. (1991). "Software Risk Management". Los Alamitos.
IEEE Computer Society Press, Los Alamitos.
Boland, R. & Hirschheim, R.A. (1985). "Series foreword in Hirschheim (1985)".
British Standard Institution (1991). "Quality Vocabulary." BS4778 (Part 3 Section
3.2=IEC 1990 50(191), BSI, London.
Buckhout, S., Frey, E. and Nemec, J. Jr (1999). "Making ERP succeed: turning
fear into promise". IEEE Engineering Management Review.
Chapman, C. & Ward, S. (1997). Project Risk Management: Processes, Techniques
and Insights. John Wiley & Sons.
Colton, K.W. (1972). "Computers and police: patterns of success and failure". Sloan
Management Review, winter, 1972-73, 75-98.
Connolly, T.M., Begg, C.E. & Strchan, A.D. (1996). Database systems: A practical
approach to design, implementation and management. Addison-Wesley.
Curtis, G. & Cobham, D. (2002). Business Information Systems: Analysis, Design
and Practice. FINANCIAL TIMES PRENTICE HALL.
Dearden, J. (1972). "Mis is a mirage". Harvard Business Review, Jan-Feb, 90-99.
Reference Lihong Zhou
- 110 -
Doke, E. R. & Barrier, T. (1994). "An Assessment of Information Systems
Taxonomies: Time to Re-evaluate?" Journal of Information technology. 9, pp149-157.
Drori, O. (1997). "From Theory to Practice or How Not to Fail in Developing
Information Systems." ACM SIGSOFT. Software Engineering Notes, vol. 22, no 1.
European Foundation for Quality Management. (2005). "EFQM Framework for
Risk Management". European Foundation for Quality Management, Brussels.
Brussels.
Finkelstein, A. (1993). Report to the Inquiry Into The London Ambulance Service.
[Online]http://www.cs.ucl.ac.uk/staff/A.Finkelstein/las/lascase0.9.pdf#search=%22R
eport%20to%20the%20Inquiry%20Into%20The%20London%20Ambulance%20Ser
vice.%22[Accessed 21/6/2006]
Flowers, S. (1996). Software Failure: Management Failure-amazing stores and
cautionary tales. John Wiley&Sons.
Flynn, D. (1992). Information System Requirements: Determination and Analysis.
McGRAW-HILL BOOK COMPANY.
Forester, T. & Morrison, P. (1994). Computer Ethics 1990. Cambridge,
Massachesette: MIT Press.
Fortune, J. & Peters, G. (1995). Learning From Failure- The Systems Approach.
Johnwy Wiley & Sons.
Frosdick, S. (1997). "The Techniques of Risk Analysis are Insufficient in
Themselves." Disaster Prevention and Management. Volume 6, Number 3, 1997,
pp.165-177.
Reference Lihong Zhou
- 111 -
Galliers, R. (1992). INFORMATION SYSTEMS RESEARCH: Issues, Methods and
Practical Guidelines. ALFRED WALLER LTD.
Gibson, J., Ivancevich, J., Donnelly, J. (2000). Organizations: Behavior, Structure,
Processes. Tenth Edition. McGraw-Hill.
Goulielmos, M. (2005). "Applying the organizational failure diagnosis model to the
study of information systems failure." Disaster Prevention and Management. Volume
14, Number 3, 2005, pp.362-377.
Gray, C. & Larson, E. (2006). Project Management : The Managerial Process. The
McGraw Hill.
Hira, R. (2003). Risk Assessment of Individual Industrial Information System Projects: A case study. University of Sheffield.
Hirschheim, R.A. (1985). Office Automation: A Social and Organizational
Perspective. Chichester: John Wiley.
Hirschheim, R. and Klein, H. (1989). "Four Paradigms of Information Systems Development". Communications of the ACM, 32 (October 1989).
Holland, C. and Light B (1999). "A Critical Success Factors Model For ERP
Implementation". IEEE Software.
Holland, P., Light, B. and Gibson, N. (1999). "A critical success factors model for enterprise resource planning implementation". Proceedings of the 7th European Conference on Information Systems, 1, 273-97. Hurst, R. (2006). How Big Is It? [Online]. www.bcentral.co.uk/marking/ebusiness/how-many-people-use-the-internet-what-do-they-use-it-for.mspx [Accessed 21/6/2006]
Reference Lihong Zhou
- 112 -
Hyde, A. (2002). "Defining IT project success and failure" British Computer Society
[Online]. http://www.bcs.org.uk/review02/html/p164.htm [Accessed 20/8/2006]
Jarvinen, P.H. (2000). "Research Questions Guiding Selection of an Appropriate
Research Method". Proceedings of the 8th EuropeanConference on Information Systems
(ECIS).
Kasser, J. (1998). "What Do You Mean You Can't Tell Me if My Project is in
Trouble?" FESMA98, Antwerp, Belgium. Antwerp, Belgium.
Kliem, R. &.Ludin, I. (1997). Reducing Project Risk. Gower Publishing Limited.
Laudon, K.C & Laudon, J.P. (1991). Management Information Systems -
Managing the Digital Firm, 5th Edition. Prentice Hall.
Lichtenstein, S. (1996). "Factors in the selection of a risk assessment method".
Information Management & Computer Security, 4/4, 20-25.
Liu, F. (2002). Critical Analysis the Factors which Contribute to IS Projects Failure in
Project Management Perspective: Three Characteristic Case Studies in the UK Public
Sectors. University of Sheffield.
Lucas, H.C. Jr. (1975). Why Information Systems Fail. New York: Columbia
University Press.
Lyytinen, K. (1988). "Expectation Failure Concept and System Analysts' View of
Information System Failures: Results of an Exploratory Study". Information &
Management, 14, 45-56.
Reference Lihong Zhou
- 113 -
Lyytinen, K. & Hirschheim, R. (1987). "Information Systems failures: a survey and
classification of the empirical literature". Oxford Surveys in Information Technology,
Vol 4, 257-309.
Lyytinen, K & Robey, D. (1999). "Learning Failure in Information System
Development". Information System Journal, 9, 85-101.
McNeill, P. (1990). Research Methods. London : Routledge.
Morgan, H.L & Soden, J. V. (1973). "Understanding MIS failures". Data Base, 2,
65-93.
Nah, F., Lau,J., and Kuang, J. (2001). "Critical factors for successful implementation of enterprise systems". Business Process Management Journal, 7 (3), 285-296.
Nickerson, R. (1986). Reflections on reasoning. Hillsdale, NJ: Lawrence Erlbaum
Associates.
O'Brien, J. (2006). Introduction to Information Systems Eighth Edition [Online].
http://www.mhhe.com/business/mis/obrien/jointis8.htm#top [Accessed 25/6/2006]
Pressman, R.S. (1987). Software Engineering: A Practitioners Approach. London:
McGraw-Hill Book Company Europe.
Pressman, R.S. (1992). Software Engineering: A Practitioners Approach - European
Adaptation. Berkshire: McGraw-Hill Book Company Europe.
Randell, B. (1992). London Ambulance Service [Online].
http://catless.ncl.ac.uk/Risks/13.88.html [Accessed 12/8/2006]
Reference Lihong Zhou
- 114 -
Robson, C. (1993). Real world Research. Oxford: Blackwell.
Robson, W. (1997). Strategic Management and Information Systems: An integrated
approach. PEARSON EDUCATION LIMITED.
Rosario, J.G. (2000). "On the leading edge: Critical success factors in ERP
implementation projects". Business World.
Royal Society (1992). "Risk: Analysuis, Perception and Management, Report of a
Royal Society Study Group". The Royal Society, London.
Sauer, C. (1993). Why Information Systems Fail: A case study approach. Alfred
Waller Ltd.
Saunders, M., Lewis, P. and Thornhill, A. (2000). Research Methods for Business
Students. Second Edition. Pearson Education Limited..
Small, F. (2000). Ministerial Inquiry Into INCIS [Online]. http://www.justice.govt.nz/pubs/reports/2000/incis_rpt/INCIS%20inquiry.pdf [Accessed 1/8/2006]
Sommerville, I. (1996). Software Engineering 5th edition. Addison Wesley,
wokingham.
Stair, R. & Reynolds, G. (1999). Principles of Information Systems: A Management
Approach. Fourth Edition. . Course Technology.
Stufflebeam, D. (2000). Guidelines for Developing Evaluation Checklists: The Checklists Development Checklist (CDC) [Online]. http://www.wmich.edu/evalctr/checklists/guidelines_cdc.pdf [Accessed 25/08/2006]
Reference Lihong Zhou
- 115 -
Sumner, M (1999). "Critical Success Factors in Enterprise Wide Information
Management Systems Projects." SIGCPR’99. , New Orleans. LA. USA. New
Orleans. LA. USA.
Sumner, M (2000). "Risk Factors in Enterprise Wide Information Management
Systems Projects". SIGCPR 2000, Evanston Illinois USA. Evanston Illinois USA.
Tchankova, L (2002). "Risk identification-basic stage in risk management".
Environmental Management and Health, 12 (3).
The Standish Group (2006). Unfinished Voyages [Online]. The Standish Group
International.
http://www.standishgroup.com/sample_research/unfinished_voyages_1.php
[Accessed 25/6/2006]
Thietart, R. et al. (2001). Doing Management Research. SAGE Publication Inc.
Trochim, W. (2006). Deductive and Inductive Thinking [Online].
http://www.socialresearchmethods.net/kb/dedind.htm [Accessed 15/8/2006]
University of Sheffield. (2000). Project Management. M3F2000
Walliman, N. (2001). Your Research Project: A Step By Step Guide For The First
Time Researcher. London: SAGE Publications.
Watters, J. J. & English, L. S. (1995). "Children's application of simultaneous and
successive processing in inductive and deductive reasoning problems: Implications
for developing scientific reasoning." Journal of Research in Science Teaching, 32(7),
699-714.
Reference Lihong Zhou
- 116 -
Wee, S. (2000). "Juggling toward ERP success: keep key success factors high". ERP
News, February.
Williams, R., Bertsch, B., Dale, B,. Wiele, T. Iwaarden, J., Smith, M. and Visser,
R. (2006). "Quality and risk management: what is the key issues?" The TQM
Magazine, 18 (1).
Wolstenholme, E.F., Henderson,S. and Gavine, A. (1993). The Evaluation of
Management Information Systems: A Dynamic and Holistic Approach. Wiley.
Wright, S. and Wright, A.H. (2002). "Information system assurance for enterprise
resource planning systems: implementation and unique risk considerations". Journal
of Information technology, 16, 5-15.
Yasin, M. M. &Quigley, J. (1994). "The Utility of Information Systems: Views of
CEOs and Information System Executives." Industrial Management & Data Systems,
94, 5, pp. 25-29.
Yeates, D. and Cadle, J. (1996). Project Management for Information Systems, 2nd
edition. Pitman.
Yin, R.K. (1994). Case Study Research: Design and Methods. London: Sage
Publications
Appendix I Lihong Zhou
- 117 -
Appendix I
Information System Project Risk Checklist Information Systems Project Risk Checklist Part 1of 5
Pre-Project 1 Complex and unclear relationship among partners, customers and suppliers
2 Disagreement between contract involved partners
3 Failure to consider existing relationships when replacing systems
4 Failure to consider existing systems when replacing systems
5 Inappropriate business processes, and reform them after the contract
6 Lack measurement system for assessing and controlling project risk before
contract
7 Lack of senior management support
8 Lack of thoroughly comprehension on organizational nature before contract
9 Project scope, objective and requirement specifications are ill defined
10 Resources are not allocated well
11 Significantly change project scope, objective and requirement specifications
after contract
12 Technical framework and development methodologies are not firmly defined in
contract
13 The customer lack of formal experience
14 Unclear payment schedule without linked milestones
15 Unsuitable business plan and vision
16 User requirements are formed only by management level without final system
users
17 Without, or ignore, backup plans for delay or under-performance
Appendix I Lihong Zhou
- 118 -
Information Systems Project Risk Checklist Part 2of 5
Customer 1 Conflicts between user departments
2 Constant external pressure and unable to properly manage them
3 Final Users are unfamiliar with the technology and require additional training
4 Inappropriately select the system supplier with ambiguous select criteria
5 Inefficient communication between involved parties
6 Internal political difficulties
7 Lack of confidence on the project
8 Lack of customer support
9 Management doesn’t understand technical issue
10 Management in the customer received over the years little or no effective
management training
11 Mistrust between management and staff
12 Reluctance of changing environment to accept the new system
13 Resources are not well assigned to the project team
14 Significantly change project scope, objectives and requirement specifications
after contract
15 The supplier is lack of experience or unsuitable for the particular project
16 Unable of unifying customers perspective on the project
Appendix I Lihong Zhou
- 119 -
Information Systems Project Risk Checklist Part 3of 5
Project Management 1 Ambiguous relationships and disagreement between involved parties after
contract
2 Client and development staff fail to attend scheduled meeting
3 Inappropriate staffing, personnel shortfalls
4 Inappropriate structure of project team
5 Inexperienced team members in business or technology
6 Lack of a complete quality assurance methodology
7 Lack of an effective methodology and poor estimation
8 Lack of effective risk identifications and risk management plan before contract,
inadequate managing risks after contract.
9 Lack of process and standard
10 Lack of, choose an unfamiliar project management methodology
11 Milestones and related deliverables are ill-defined
12 No single person is accountable/responsible for the project
13 Team members are not fully committed to the project
14 Too many junior staff or too many senior staff
15 Unfamiliar environment for system developers
16 Unrealistic and inflexible timeliness and budgets
Information Systems Project Risk Checklist Part 4of 5 Project Technical Aspect
1 Coding from high level requirement without thoroughly design
2 Constant close partnership between the customer and the supplier after the
project
3 Emerging or unproven, and inappropriate developing technologies and
methodologies
4 Significantly change technical infrastructure after system design
5 Unfamiliar developing environment to the developers
6 Unfamiliar developing technologies or methodologies for system developers
7 Unstable or incompatible software and hardware system platform
Appendix I Lihong Zhou
- 120 -
Information Systems Project Risk Checklist Part 5of 5
Project Development Development
Stages
ID Risk Factors
1 Incomplete comprehension of the current system and current situation
of the organization
2 Emerging or unproven, and inappropriate developing methodologies
and technologies
3 Partially understand the user requirement
4 Significantly change project scope, objectives and requirement
specifications
System
Analyzing
and
Requirement
Defining
5 User requirement are formed only by management level without final
system users
6 Inadequate initial general planning before detailed and complex
designing
7 Unstable or incompatible software and hardware system platform.
8 Too accurate designing causes the system extremely heavy and
unhealthy
9 Without an agreed prototype by the customer
System
Designing
10 Poor communication and understanding among the system analyst,
designer and the programmer
11 Attempting produce a complete and perfect system in one stage
12 Initiate system programming when the system designing is not
completed
13 Accept testing without final users involvement
14 Inadequately test the final integrated system before final
implementation
15 Programming without a preselected and unified methodology
16 Programming without necessary documentations
17 Lack of considering reuseable components from the current system
System
Developing
and
Acceptance
Testing
18 Programmer-oriented system designing instead of user-oriented
19 Lack of new system cutover methodologies
20 Lack of a user training plan and insufficient user training
System
Training
21 Initiate training without complete testing
22 Lack of a thorough maintenance plan System
Maintaining 23 Assign unqualified staff for system maintenance
Appendix II
- 121 -
Appendix II
An Example of WBS diagram of an information systems development: