1

Failure is a tricky thing to categorise though generally defined in terms of cost, schedule and

scope, i.e. the iron triangle.

However often a seemingly successful project is overwhelmingly viewed as a failure (Millennium

Dome) while an on-paper unsuccessful project is seen as successful (London Eye).

Both were designed for the new millennium, January 1st 2000.

The Millennium Dome was opened on time and within budget. Architect, Richard Rodgers,

stated that ‘I can’t think of a project that went better’.

The London Eye suffered technical problems and opened three months late but is ‘one of the

rare few London Millennium projects which has been a great success’.

Part of the Literature review’s scope was to look for causes and recommendations.

2

Several surveys and studies have identified complexity as a contributory factor in project failure.

Even when not mentioned explicitly, the issues can often be indirectly attributed to complexity

– McKinsey survey shown.

Complexity is generally increasing and is linked to the prevalence of the ‘wicked problem’ – that

is a problem that is ‘highly resistant to resolution’

How do we address this?

The Literature survey included reviews of complexity theories and models across a wide variety

of domains.

3

Non of the mainstream project management or systems engineering methodologies directly

address complexity. There are however a few specialised organisations that recognise the

challenges of complexity as a discrete entity.

The most common approach is that of decomposition of a large and complex projects within

existing methodologies.

Organisations looking to address complexity include:

• International Centre for Complex Project Management – uses Helmsman Institute’s

assessment

• US’s Transportation Research Board

Literature survey looked at how complexity is currently managed and the project looked at

potential shortfalls.

Ultimately the project reconciled the various approaches to how complexity is understood

and modelled and combined techniques and models into an assessment.

4

Complexity introduces or contributes to issues and risks.

Design a framework that will anticipate the likely issues that will be encountered and allow

better risk management. Use the Demming Cycle as a basis.

• Complexity assessment to identify areas of concern;

• Identify Critical Success Factors to address the findings of the Complexity Assessment and

show areas of risk or criticality to receive additional effort ;

• Planning of scope with techniques tailored accordingly and effort according to risk and

criticality;

• Assignment of Performance Measures according to the characteristics of the scope;

• Identify residual risks;

• Repeat at prescribed intervals to ensure continued validity and adjust approach as necessary.

Ultimately the approach was to identify where effort would be best expended and to dovetail

into existing project and engineering management practise.

Components of framework were taken from existing literature.

Sub-processes (CSFs and Performance Measures) were subject to a questionnaire survey to

allow them to be validated and ranked according to influence.

Case studies were used to demonstrate how the technique would be used in practice and

identify shortfalls and areas for development.

Framework would be compared against literature to assess how well it met the

recommendations that had previously been identified.

5

Work Breakdown Structure (WBS) or PBS – standard activity to determine this at beginning, and

develop during, project lifecycle;

Complexity Themes look at how aspects of the project may be divided into 9 parts as follows:

1. Internal factors

• (Project) environmental constraints – external factors (inputs and outputs only),

schedule (such as pace and phased handover requirements), the imposed funding

profile;

• Development process – how prescriptive and many interfaces;

• Internal organisation – ranging from few disciplines within same organization to

many and organisation type;

2. External factors

• Contractual management – ranging from a few simple relationships to many

relationships with differing contractual terms;

• Stakeholders - relating to system definition and validation;

• Regulatory interfaces – how many and how influential they are on the

development;

3. System development

• External (system) interfaces – the degree to which system impacts or depends

upon external systems (from none to part of system of systems);

• Technology – system requirements definition and verification relating to

technology type (low to very high tech and low to highly novel);

• Internal (system) interfaces – the level of internal integration required between

6

sub-systems ranging from few simple relationships to many complicated

relationships.

Complexity Criteria decompose complexity into five parts:

• Uncertainty – likelihood of unexpected events occurring, e.g. stakeholder requirements

change significantly during system early development or assumptions are found to be

incorrect;

• Ambiguity – incompleteness of knowledge about functional variables [25], i.e., a lack of

information upon which to make decisions. An example may be that stakeholder requirements

are undefined in a particular area of system development leading to assumptions being made

or to delays in the development schedule while the stakeholder requirements are defined;

• Emergence – how change to system configuration leads to unexpected behaviours and

interactions and re-evaluation of derived system requirements. High Emergence is defined as

unexpected

• behaviours and interactions in many other components or sub-systems;

• Non-linearity – the effect of a single small change to system requirements on other system

requirements. A non-linear relationship is one where a change results in a disproportionate

change in affected components and sub-systems. High Non-linearity is where a small change

results in a large impact elsewhere;

• Program-size Complexity – relates to the minimum amount of information required to

describe a process, system or organisational requirements. Manifests itself in shortfalls in

modelling and a general low level of understanding in the system or its development even

when uncertainty and ambiguity is low. High Program-size Complexity requires a high level of

effort to model and to maintain the model. Examples where this may be manifested are

requirements management, configuration management and scheduling.

6

CSFs chosen from literature and assessed within a questionnaire survey for validity and strength

of influence – 122 respondents across various age groups, job descriptions and industries.

Measurement of efficacy of CSFs is in the spirit of the Demming Cycle.

CSFs were assessed against age and role to identify biases in CSF selection and industry for the

potential for tailoring for specific industries.

Print off summary of findings.

7

Balance effort versus benefit

• Only use where significant risk, criticality or complexity has been identified;

• Binary and numerical DSMs – complexity and effort with regards planning should be

commensurate with the risk being mitigated;

Integrate with Gantt chart usage rather than compete with it

• Relationship types translate between two tools;

• Outputs from DSM can be used for Gantt Charts;

• Tooling is under-developed.

Performance Measures can be derived relating to Complexity Criteria and be used to assess

plan complexity (leading measure).

8

Performance Measure chosen from literature and assessed for validity and strength of influence

in questionnaire survey.

Can use Goal-Question-Metric technique to determine what should be measured.

Numerous types of measures were considered with the top three being:

• Measurement of requirements process such as requirements status (created, approved,

verified, validated) and other attributes such as owner, trace to stakeholder, risk etc.

• Measurement of the realisation of stakeholder and system requirements (Measure of

Effectiveness, Measure of Suitability, Measure of Performance and Technical Performance

Measure) during the project lifecycle.

• Measurement of plan complexity through determination of proportion of sequential,

parallel, coupled and conditional activities on the schedule

Earned Value Management was eighth – this is the most popular and often used in isolation,

demonstrating general dissatisfaction with the technique.

Print off of summary of findings.

9

This ties all the sub-processes together.

The framework was assessed against literature to adjust whether it had the potential to meet

recommendations.

Also assessed at a low level of detail against two case studies – Boeing’s Dreamliner and Areva’s

OL3 project.

10

Inconsistently written WBS can be problematic – good to have early involvement and influence

this activity;

Also have to be aware that decisions made early have an effect later on – i.e. contractual

strategy. Early is better.

Complexity Assessment has little value alone but provides a way of determining areas of concern

for CSFs, planning and Performance Measures.

Complexity Assessment is subjective so needs guidance to ensure greater consistency of

application – also have to take care not to provide too high a score early on leaving no scope to

increase it later.

Selection of CSFs would benefit from development – scoring of efficacy of CSFs against each

Complexity Criteria to make selection easier;

May be resistance to using the framework due to the additional resource overhead – emphasis

on targeting complexity to give most bang for buck.

Can apply sub-processed in isolation for additional development, particularly CSFs and

Performance Measures. Additional value through use of full process.

Feedback of learning important to improve the processes and lists of CSFs and Performance

Measures for use;

Practical application of full process or sub-processes is required to develop it to its full potential.

11

12