Part B - Information Systems and Information Technology Solutions

B.4.9. Criteria for Appraisal of the Financial Investment in IS&T Projects

When investing in IS&T, a number of key criteria for evaluating and approving the project must beconsidered. Investments in IS&T are no different to other significant investments in terms of theprocedures they must follow and the need for rigorously constructed business plans. However, themixture of technical and organizational issues raised by such investment demand that severalquestions must be raised by the decision-makers involved in the process of systems planing, design,and acquisition. They form the basis for criteria against which approval will be given. The criteriaapply irrespective of the source of financing.

The discussion that follows is based on recommended criteria used by the U.K. National HealthServices, developed by the NHS Management Executive, Information Management Group. Criteriawhich management should apply in assessing all IS&T investments, and against which approval willbe given for those which are substantial, are:

1. The investment is part of an overall IS&T strategy based on the organization's business plan

Every IS&T investment must support the business and service objectives of the organization. Theseare usually detailed within the organization's business plan. It is also important that the total of all theIS&T systems within an organization provide consistent and coherent support to the business. It is,therefore, essential that every investment is part of an overall IS&T strategy based on theorganization's business plan. Strategic horizons will generally be three to five years with an annualreview.

2. There is a properly structured business plan based on good investment appraisal and realistic scheduling, with staged review points

When scrutinizing a business plan one must expect that the appropriate development of a ProjectPlan has been considered according to the criteria defined in Sections B.2.1. to B.2.8. above.

3. Account has been taken of achieving the same benefits from better use of existing assets

A range of IS&T options will usually exist, delivering differing levels of benefit but at differing costsand must be considered within the investment appraisal. The following options must be considered:

(a) Status quo - what would happen if the investment did not proceed? This must always beincluded for comparison purposes.

Part B - Information Systems and Information Technology Solutions

(b) The non-IS&T solution - is it possible to alter current practices to achieve increased benefitswithout implementing an IS&T solution?

(c) Changing existing systems - is it possible to increase the benefits from existing systems bychanging the systems or the organization's working practices rather than implementing a newsystem?

4. The benefits (cash-releasing and noncash-releasing) have been properly and realistically identified and assessed with a commitment from the affected parties to their realization

In order for a benefit to be properly and realistically identified and achieved, it is necessary that:

(a) the benefit is accurately and precisely described;(b) methods for measuring the benefit are outlined;(c) the tasks required to achieve the benefit are identified; and(d) a commitment to the achievement of the benefit is obtained from those affected.

If an investment in a provider organization is likely to result in increased costs to purchasers, then thecontinued viability of the provider's business plans should be considered. National demonstrationprojects may be of help in identifying the benefits that can be achieved.

5. Full assessment of the risks surrounding the investment is carried out at an early stage together with an evaluation, setting out how sensitive options are to change in the underlying assumptions that have been made

Options are often close together in terms of costs and benefits and the estimates used in the optionappraisal can be uncertain. It is important to perform an analysis of these uncertainties and twoessential elements sensitivity analysis and risk analysis must be addressed.

Sensitivity analysis will identify the assumptions made and consider the impact of varying theassumptions made on the options. Risk analysis considers the nature of the risks associated with theinvestment and their impact on the project. What measures can be introduced to minimize theirimpact?

6. There is a clear understanding of the procurement process

The procurement process will vary depending on the type, complexity, and size of the project beingundertaken. In all cases the procurement process and the proposed contract will be expected tofollow the guidelines explained in detail in Part C of this document. The contract must conform to allthe necessary contractual safeguards.

Part B - Information Systems and Information Technology Solutions

7. The project will be handled in a structured manner

Each major project must be managed using a well-structured methodology. Guidelines can be foundin a number of easily available general publications related to technology project management. Ageneral introduction and recommendations are found in Sections B.2.1. to B.2.8. above. For largeinvestments, consideration should be given to appointing a Project Supervisory Committee.

All participants in the project management structure must have an appropriate level of training and/orawareness of their roles. The Project Manager must be fully trained and experienced in informationsystems project management, and projects should not be started unless this can be demonstrated.

8. There is an unequivocal commitment from the Project Manager and Project Supervisory Committee and clear understanding of their roles, and of their senior staff, in the procurement, implementation, and benefits realization process

Implementation of a major IS&T investment can have a major impact on an organization. Seniormembers of the organization must be aware of all the potential internal and external impacts and becommitted to the successful achievement of the investment.

9. There are sufficient and adequately skilled IS&T resources to manage successfully the specification, procurement, and implementation of the project

The IS&T human resources requirement must be identified. Where such resources are not availablewithin the organization, the methods for obtaining the resources must be identified. When resourcesare obtained from outside the organization (outsourcing) it is important to ensure that the organizationhas sufficient skills to absorb their contribution and supervise those resources.

This will be an area of particular scrutiny. It can be difficult to recognize that in-house skills andexperience may not be sufficient. However, this will be critical to a project's success. The larger theproject the more skilled and experienced the project manager and internal IS&T staff need to be. It isthe responsibility of the organization to make decisions on the suitability of their IS&T staff.

10. There is a resourced and structured training program

Implementing a major IS&T investment will mean training staff to operate the new procedures. Atraining need assessment should be performed and a structured training program produced to meetthe needs identified. The plan for achieving benefits will normally be described within the businesscase. This will often be a substantial part of the investment up to 15% of the capital costs.

Part B - Information Systems and Information Technology Solutions

11. There is a clear plan for benefits realization, including a commitment to assign responsibility for realizing benefits to an individual with sufficient authority and resources to deliver

In order to obtain benefits from an IS&T investment, changes to the organization and its workpractices will often be required. To successfully implement the investment requires that an individualwith sufficient authority and resources be prepared to commit to achieving the benefits outlined. Toensure that all parties play their role in achieving benefits, that authority should normally derivedirectly from the Project Manager.

As benefits are normally obtained within user departments or units, the managers of thosedepartments, rather than IS&T specialists, will normally be responsible for the actual achievement ofbenefits. It is the responsibility of the Project Manager to ensure that they do so.

12. There is a commitment to post-implementation evaluation, the results of which will be made available to the authority approving the investment

It is essential to have a formal post-implementation review program whose prime purpose should beto identify problems and initiate appropriate remedial action. The process should not be regardedsolely as a single exercise that occurs once the project has been implemented, but should be basedon a continuing process which contributes to a final formal review at an appropriate point. This reviewshould be aligned to a suitable milestone in the achievement of benefits, and should occur within twoyears of implementation. Plans for collecting monitoring and evaluating data should be set out in thepreparation phase of the business case.

Where projects take a year or more to implement from the signing of the contract, or if there is nocontract one should consider the start of implementation, the organization should identify a significantmilestone in each year. At this milestone the Project Committee should review the project to ensure itis on schedule, within costs and that it should continue. A report should be submitted to the normalapproval body. For projects where implementation will take a year or less, a post-implementationreview report should also be submitted at the end of the scheduled implementation.