UK Public Sector IT Procurement

27
UK Public Sector IT Procurement Of shoes and ships and sealing wax, and whether pigs have wings or… Can IT procurement ever work better than it does?

description

An overview of issues in public IT project delivery, their causes and possible mitigating strategies. Presented to the National Audit Office.

Transcript of UK Public Sector IT Procurement

UK Public Sector IT Procurement

Of shoes and ships and sealing wax, and whether pigs have wings

or…Can IT procurement ever work better

than it does?

What are the issues?

Too many IT projects run late, over-budget and fail to deliver “You can always tell the expert in the crowd – he’s the

one who says it will take longest and cost the most.” (Murphy’s Laws of almost everything)

Is it just bad luck? If projects continue to fail despite the application of

management methodologies and processes what else is there?

If there are specific causes, can they be avoided?By formal methods or otherwise

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 2

IT procurement failures

Government IT spend 2003-2004 = £12.5bn Of which, central government spending = £3bn

Failures achieve very high prominence Computer Weekly described NAO’s report on the Libra project as

“damning” (Sir) David Omand, then Permanent Secretary at the Home office

described the Immigration and Nationality Directorate's Casework Application programme as “in effect a doomsday machine” and the cancellation featured in the media for weeks

Public sector failures are more prominent Because the government is accountable to parliament and the

taxpayer – and its failures attract strong media interest But there are also private sector failures

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 3

Uncommon senseThe Unknown

As we know, There are known knowns. There are things we know we know. We also know There are known unknowns. That is to say We know there are some things We do not know. But there are also unknown unknowns, The ones we don't know We don't know.

Donald Rumsfeld, Feb. 12, 2002, Department of Defense news briefing

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 4

What do we know?

We know we knowThe objectiveThe planned routeHow to follow the route

We know we don’t knowWhat changes may occur to the objective, and what

changes to the route they will entailHow well we can follow the routeWhat other changes to the route may be required

We don’t know Anything else

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 5

Consequences

What we know can be formalised and managed Determining the objective, the route and how to follow it is the

domain of methodologies and processes Change can be managed but not controlled

Formal methods propagate the effects of change in a controlled manner, but cannot tell us what those effects will be

If the route is blocked, whilst one may have criteria for selecting an alternative, the choices presented are not inherent in the system

Conclusion The problem is one of dynamics – how confronting changes

causes changes – and how we cope with the unforeseen

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 6

Message of the day

Project management is replete with rules Rules encapsulate knowledge of regularities

gleaned from prior endeavours “Experience is what you get just after you needed it”

(Murphy’s Laws of almost everything, again) But they are not enough

The best strategy would be to learn how to work within and without the rules

Can we create a strategy for working outside the rules? …and IT is not a goal in itself – it supports

business change and development

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 7

What is generally right?

Methodologies etc.PRINCE2, ITIL

ProcessesGateway Reviews

OGC guidance generally Impressive output

Standards ISO 9001 for design, development, production,

installation and/or servicing

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 8

What is less right? Process detail

The stages between gateways Especially between Gateways 3 & 4 (“Investment Decision” and

“Readiness for Service”) – delivering the IT And between Gateways 0 & 1 (“Strategic Assessment” and “Business

Justification”) – establishing the business case Attitudes to methodologies

“Lip service” to PRINCE2 etc. Quality assurance

Recognition that there is a substantial and important difference between defined quality and high quality

Development approach too linear Less “big bang”, more iterative and scalable developments, use of

“model office” environments

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 9

Specific contributory weaknesses

Governance Inappropriate and ineffective decision-making

Risk, issue and change managementRisk is not transferred, “ownership” of risk and issues is

indeterminate, risk and issues are neither understood nor well managed; change is “managed” not confronted

Application of methodologies and processesMinimal compliance

People, their abilities, relationships and culturesThese are areas of difficulty where formalisation is

impracticable and where leadership is most needed

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 10

Weaknesses of governance

Lack of clear management structuresEspecially roles & responsibilities

Lack of clear Terms of ReferenceEspecially relating to automatic escalation according to

defined Risk/Issue criteria Lack of appropriate authority

And willingness to exercise it An indirect function of people

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 11

Weakness of risk, issue & change management

Risk is not transferredSuppliers establish their margins as a function of their

success rate and recover costs from previous failures on subsequent projects They should improve with time, but someone else will get the

benefitRisk management for suppliers includes resistance to

minor changes and willing acceptance of larger changes that lead to non-competitive re-negotiation

Change is a “known unknown”But is not catered for

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 12

Weaknesses of application

Lack of appreciation for the value of well maintained systems and project documentationDocumentation must be intelligible to external readers

Emphasis on the letter rather than the spiritPIDs are “rote documents”

Documentation weak and rarely currentMaintaining documentation is not seen as a contribution

to current work – it is done afterwards An indirect function of people

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 13

Weaknesses of people – ability

The average is average IT projects are no more likely to be completed on time,

to budget and to requirements than An extension to your house The Christmas shopping

Responsibility & AuthorityKnowledge and know-how do not necessarily correlate

with a willingness to accept responsibility and exercise authority – even if “responsibility” is part of the role

Even if the willingness is there, adequate and effective authority may not be delegated

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 14

Weaknesses of people – relationships

“The Customer is Always Right” No, pleasing the customer is not the objective on all

time-scales: the goal is to achieve a specified business benefit Consider general optimisation methods, to reach a new

optimum one must pass through places that are less good; if one cannot move away from a local maximum one cannot reach any other

Lack of respect and trust “Wreckers” – passive rather than active opposition:

those whose support is required for success do not give it because they believe their efforts will be wasted; they do not trust the strategy/approach etc.

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 15

Weaknesses of people – culture

Asymmetric CulturesBusiness is used to monthly, quarterly and annual

performance objectives with rewards for success and penalties for failure

The public sector does not generally place the same emphasis on timeliness

Business is used to leading and responding to market developments

The public sector is accustomed to working within the more rigid constraints of legislation and policy – it may be considered less flexible

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 16

Governance – thoughts

Governance structures etc. should be proposed in the “Procurement Strategy”

Bids should specifically address governance issues

Contract negotiation should include initial roles, responsibilities and authorities…and key named individuals: a project should not be a

repository for unwanted business personnel

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 17

Risk, issue & change management – thoughts

Investigate and develop change metrics What are the incidence and costs of changes to specifications

through the life of a project, as e.g. a function of the number of changes to specification/requirement line items (by level)?

Recognise that change is a major source of risks and issues Do audit trails exist, sufficient to be able to compare the

specification as implemented with that originally established? Standardise risk and issue metrics

e.g. five probability bands, five impact levels Include risk & issue assessments in bids – and require bidders to

make specific provisions for avoidance and mitigation

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 18

Application – thoughts

Require “no notice” auditsCarrot & stick –reward for superior performance,

penalties for inadequate performance Make the criteria relative e.g. bottom quartile performance

leads to penalty, upper quartile performance leads to reward; lesser deficiencies and achievements carry forward

This could be made self-financing if more is taken in penalties than is paid out as rewards

Whether a particular audit is to be wide ranging or focussed, the possibility of either requires all areas to be properly maintained

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 19

Other broad issues

Auxiliaries of porcine aviation

Communications - the importance of being earnest

Earnestadjective intent; sincere in intention; serious in

disposition; determined or whole-hearted; fervent or impassioned

UnearnestnessLeads to poor communicationPoor communication creates and sustains ambiguity,

confusion and error “Management-” or “Consultant-” speak is not earnest –

it is euphemistic Death to “expectation management”!

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 21

The bidding process

Competition is “all or nothing” It’s a set menu, not à la carte

Questions – would it be possible to…?Require bids to be anonymised for

Technical evaluation Risk assessment

Only identify bidders to assess their costs and risk carrying capacity

Bear in mind that “supplier’s staff” may be sailing under flags of convenience

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 22

The interplay of business and ITOne fine winter's day when Piglet was brushing away the snow in front of his house, he happened to look up, and there was Winnie-the-Pooh. Pooh was walking round and round in a circle, thinking of something else, and when Piglet called to him, he just went on walking.

"Hallo!" said Piglet, "what are you doing?""Hunting," said Pooh. "Hunting what?" "Tracking something," said Winnie-the-Pooh very mysteriously. "Tracking what?" said Piglet, coming closer "That's just what I ask myself. I ask myself, What?" "What do you think you'll answer?" "I shall have to wait until I catch up with it," said Winnie-the-Pooh.

Business and IT are in constant flux – if your vision is rigid, you will probably never see it realised. Nor will any fixed route get you to your destination – you need options, alternative routes; organisations should be networks not chains of command, ideally Scale Free Networks.

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 23

Scale Free Networks

What is a scale free network? A scale-free network is a network in which some nodes are hubs –

nodes that are "very connected" – and overall connectivity is described by a power law

So what? Most organisations are not scale free: information and knowledge

flows slowly – if at all – between units The more knowledge is shared, the greater the understanding and

the fewer the problems The Smoking Room

If you want to know what’s happening on a project or in an organisation, go to the smoking rooms (where still lawful) or stand outside with them.

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 24

Fundamental issues

Project InertiaCompare a project with driving a stage in the driving

rally: The co-driver uses “pace notes” to describe to the driver the

course section immediately ahead so he can drive it quickly but safely

Pace notes may indicate the tightness of an approaching bend, its direction and an appropriate gear. It takes time to prepare a change of speed and direction and there is a limit to the speed at which the bend can be safely taken

Trying to take an unexpected corner (risk or issue) may be beyond the grip and turning ability of the car or driver – expect to overshoot or crash

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 25

General suggestions Maintain sources of independent advice

The Public Sector should be willing to spend a considerable sum to maintain a pool of particularly able individuals

To audit, conduct “red team” reviews Let them rove to create a scale free network

To mentor and advise and bring the benefit of skills and experience Reduce timescales between deliveries

Developments should deliver to the business at relatively short intervals

Embrace business dynamics Increase use of dynamic simulation modelling

Embrace the co-evolution of processes & IT Remember that the best systems are open systems …and that enforced homogeneity of systems and processes stifles

development

07/04/23 NAO/RAND Europe - IT Projects - Copyright (C) 2004 Julian Moore, [email protected] 26

Can pigs fly?

Yes, but not without help