This Report is part of the requirement for the Master degree in...

This Report is part of the requirement for the Master degree in Software Engineering

Research Project

Software Development Methodologies in Industry

Australian National University

June 2010 Ahmad Deeb

Research project Software Development Methodologies in Industry

- i -

Table of Contents

Software Development Methodologies in Industry........................................................................ i

Acknowledgements...................................................................................................................... 8

CHAPTER 1 ...................................................................................................................................... 9

Introduction ....................................................................................................................................... 9 Purpose and Scope ...............................................................................................................10 Objective and mission implementation................................................................................11 Terminology.........................................................................................................................12

CHAPTER 2 .................................................................................................................................... 13

Background of Software Development Lifecycles ...................................................................... 13 Waterfall...............................................................................................................................13 Spiral ....................................................................................................................................15 Incremental...........................................................................................................................16 Agile development methodologies.......................................................................................17

Extreme Programming (XP) .........................................................................................17 Scrum ............................................................................................................................18 Dynamic Systems Development Method (DSDM) ......................................................19 Adaptive Software Development ..................................................................................21 Crystal ...........................................................................................................................22

CHAPTER 3 .................................................................................................................................... 23

Research Methods ........................................................................................................................... 23 Introduction ..........................................................................................................................23 Description of the most common research methods ............................................................23 Selected research type ..........................................................................................................24

Survey (Questionnaire)................................................................................................................... 25 Questionnaire objectives ......................................................................................................26 Questionnaire design............................................................................................................26 Conducting the Survey (questionnaire)................................................................................27 Responses expectation and alternatives ...............................................................................28

Some feedback ..............................................................................................................28 Risk management..........................................................................................................28

CHAPTER 4 .................................................................................................................................... 29

Results Analysis, Conclusion and Future study .......................................................................... 29 Statistical figures ..................................................................................................................29 Analysis................................................................................................................................30


- ii -

Organisational culture, Lifecycles and Techniques .............................................................31 Individual project investigation by SDLC ...........................................................................36

Waterfall lifecycle.........................................................................................................38 Project complexity ......................................................................................................43 Rate of change in the projects and constraints............................................................49 Project criticality.........................................................................................................53 Team experience and communication ........................................................................55

Findings – Waterfall lifecycle.......................................................................................65 When to use and not to use waterfall, based on the analysis ......................................67

Agile lifecycles .............................................................................................................69 Projects description.....................................................................................................69 Project complexity ......................................................................................................74 Rate of change in the projects and constraints............................................................78 Project criticality.........................................................................................................81 Team experience and communication ........................................................................83

Findings – Agile methods .............................................................................................93 When to use and not to use Agile methods based on the research data......................95

Incremental lifecycle.....................................................................................................96 Project complexity ....................................................................................................100 Rate of change in the projects and constraints..........................................................103 Project criticality.......................................................................................................107 Team experience and communication ......................................................................109

Findings – Incremental lifecycle.................................................................................118 When to use and not to use incremental based on the research................................119

Iterative lifecycle ........................................................................................................120 Project complexity ....................................................................................................125 Rate of change in the projects and constraints..........................................................128 Project criticality.......................................................................................................131 Team experience and communication ......................................................................133

Findings – Iterative lifecycle ......................................................................................141 Spiral lifecycle ............................................................................................................142

Investigated Spiral Projects ......................................................................................144 Project complexity ....................................................................................................146 Rate of change in the projects and constraints..........................................................148 Project criticality.......................................................................................................151 Team experience and communication ......................................................................152

Findings – Spiral lifecycle ..........................................................................................159 Conclusions ....................................................................................................................................160

Future research & study .....................................................................................................163


- iii -

Appendix A ....................................................................................................................................165 Software Development Methodologies user table .............................................................165

Appendix B.....................................................................................................................................167 Questionnaire .....................................................................................................................167

Appendix C ....................................................................................................................................184 Questionnaire design..........................................................................................................184

Appendix D ....................................................................................................................................186 Email template ...................................................................................................................186

Appendix E.....................................................................................................................................188 Questionnaire Feedback sheet............................................................................................188

Appendix F.....................................................................................................................................189 Figures (SDM background – section 2) .............................................................................189

Appendix G ....................................................................................................................................195 Organisation culture graphs and related tables ..................................................................195

Appendix H ....................................................................................................................................203 Participants’ responses on question 50 in the questionnaire ..............................................203

Waterfall projects........................................................................................................203 Agile projects ..............................................................................................................205 Incremental projects....................................................................................................206 Iterative projects..........................................................................................................207 Spiral projects .............................................................................................................207

Appendix I......................................................................................................................................208 Agile methods roles............................................................................................................208

Appendix J .....................................................................................................................................209 Questions support analysis .................................................................................................209

Appendix K ....................................................................................................................................211 Participants comments .......................................................................................................211

Appendix L.....................................................................................................................................212 Systematic approach...........................................................................................................212

References.......................................................................................................................................213


- iv -

Table of Figures

Table 1: SDLC by number of organisation and projects ...................................................................... 30 Figure 11: Number of SDLC with organisation size ............................................................................ 32 Table 8: Summary result for questions 7 to 12..................................................................................... 36 Table 9: Combination usage of SDLC ................................................................................................. 37 Figure 24: Investigated SDLC by individual project............................................................................. 37 Figure 25: Waterfall – determine the completion of projects................................................................ 42 Figure 26: Waterfall – determine the completion of projects by defects .............................................. 42 Table 11: Waterfall – Project complexity first criteria......................................................................... 48 Figure 27: Waterfall – Project complexity second criteria.................................................................... 48 Figure 28: Waterfall – Projects requirements defined at the beginning of the project .......................... 49 Figure 29: Waterfall – Rate of change in projects by request ............................................................... 51 Figure 30: Waterfall – Level of quality expected in the projects .......................................................... 51 Table 13: Waterfall – Investigated project status and duration ............................................................ 52 Table 14: Waterfall – Expected number of defects at the completion of project ................................. 53 Table 15: Waterfall – How critical is the failure of project to the organisation ................................... 53 Figure 31: Waterfall – Projects budget.................................................................................................. 54 Figure 32: Waterfall – Project team’s experience ................................................................................. 55 Figure 33: Waterfall – Team empowerment.......................................................................................... 56 Figure 34: Waterfall – Frequent team meeting (fortnightly) ................................................................. 57 Figure 35: Waterfall – Common communication methods among team members ............................... 58 Figure 36: Waterfall – Common communication methods between team members and managers...... 59 Figure 37: Waterfall – Project Management/Leadership main roles and experience............................ 60 Figure 38: Waterfall – Project Management/Leadership experience .................................................... 61 Figure 39: Waterfall – Techniques used in projects .............................................................................. 62 Figure 40: Waterfall – Advantages/benefits of the used lifecycles ....................................................... 63 Figure 41: Waterfall – Disadvantages/limitations of the used lifecycles .............................................. 64 Table 17: Agile & Techniques (by project and organisation sizes)...................................................... 71 Figure 42: Agile – determine the completion of project........................................................................ 72 Figure 43: Agile – determine the completion of projects by defects..................................................... 72 Table 18: Agile - Project complexity first criteria................................................................................ 77 Figure 44: Agile – Project complexity second criteria .......................................................................... 77 Figure 45: Agile – Projects requirements defined at the beginning of the project ................................ 79 Figure 46: Agile – Rate of change in projects by request...................................................................... 79 Figure 47: Agile – Level of quality expected in the projects ................................................................ 80 Table 20: Agile - Investigated project status and duration ................................................................... 81 Table 21: Agile – How critical is the failure of project to the organisation ......................................... 81 Figure 48: Agile – Projects budget ........................................................................................................ 82 Figure 49: Agile – Project team’s experience ....................................................................................... 83 Figure 50: Agile – Team empowerment................................................................................................ 84 Figure 51: Agile – Frequent team meeting (fortnightly) ....................................................................... 85


- v -

Figure 52: Agile – Common communication methods among team members ..................................... 87 Figure 53: Agile – Common communication methods between team members and managers............ 88 Figure 54: Agile – Project Management/Leadership main roles and experience .................................. 89 Figure 55: Agile – Project Management/Leadership experience .......................................................... 89 Figure 56: Agile – Techniques used in projects .................................................................................... 90 Figure 57: Agile – Length of iteration (weekly).................................................................................... 91 Figure 58: Agile – Advantages/benefits of the used lifecycles ............................................................. 92 Figure 59: Agile – Disadvantages/limitations of the used lifecycles .................................................... 93 Table 24: Incremental & Techniques (by project and organisation sizes) ............................................ 97 Figure 60: Incremental – determine the completion of project ............................................................. 98 Figure 61: Incremental – determine the completion of projects by defects........................................... 98 Table 25: Incremental - Project complexity first criteria.................................................................... 102 Figure 62: Incremental – Project complexity second criteria .............................................................. 103 Figure 63: Incremental – Projects requirements defined at the beginning of the project .................... 104 Figure 64: Incremental – Rate of change in projects by request ......................................................... 105 Figure 65: Incremental – Level of quality expected in the projects .................................................... 106 Table 27: Incremental - Investigated project status and duration....................................................... 107 Table 28: Incremental – How critical is the failure of project to the organisation ............................. 107 Figure 66: Incremental – Projects budget............................................................................................ 108 Figure 67: Incremental – Project team’s experience ........................................................................... 109 Figure 68: Incremental – Team empowerment.................................................................................... 110 Figure 69: Incremental – Frequent team meeting (fortnightly) ........................................................... 111 Figure 70: Incremental – Common communication methods among team members ......................... 112 Figure 71: Incremental – Common communication methods between team members and managers.............................................................................................................................................. 113 Figure 72: Incremental – Project Management/Leadership main roles and experience...................... 114 Figure 73: Incremental – Project Management/Leadership experience .............................................. 115 Figure 74: Incremental – Techniques used in projects ........................................................................ 116 Figure 75: Incremental – Advantages/benefits of the used lifecycles ................................................. 117 Figure 76: Incremental – Disadvantages/limitations of the used lifecycles ........................................ 117 Table 30: Incremental – When to use and not to use incremental methods ....................................... 119 Table 31: Iterative & Techniques (by project and organisation sizes) ................................................ 122 Figure 77: Iterative – determine the completion of project ................................................................. 123 Figure 78: Iterative – determine the completion of projects by defects .............................................. 124 Table 31: Iterative - Project complexity first criteria ......................................................................... 126 Figure 79: Iterative – Project complexity second criteria.................................................................... 127 Figure 80: Iterative – Projects requirements defined at the beginning of the project.......................... 128 Figure 81: Iterative – Rate of change in projects by request ............................................................... 129 Figure 82: Iterative – Level of quality expected in the projects .......................................................... 130 Table 33: Iterative - Investigated project status and duration............................................................. 130 Table 34: Iterative – How critical is the failure of project to the organisation................................... 131 Figure 83: Iterative – Projects budget.................................................................................................. 131 Figure 84: Iterative – Project team’s experience ................................................................................. 133 Figure 85: Iterative – Team empowerment ......................................................................................... 134


- vi -

Figure 86: Iterative – Frequent team meeting (fortnightly)................................................................. 135 Figure 87: Iterative – Common communication methods among team members ............................... 136 Figure 88: Iterative – Common communication methods between team members and managers ..... 137 Figure 89: Iterative – Project Management/Leadership main roles and experience............................ 138 Figure 90: Iterative – Project Management/Leadership experience .................................................... 138 Figure 91: Iterative – Techniques used in projects.............................................................................. 139 Figure 92: Iterative – Advantages/benefits of the used lifecycles ....................................................... 140 Figure 93: Iterative – Disadvantages/limitations of the used lifecycles .............................................. 141 Table 36: Spiral & Techniques (by project and organisation sizes) .................................................... 143 Figure 94: Spiral – determine the completion of project..................................................................... 144 Figure 95: Spiral – determine the completion of projects by defects .................................................. 145 Table 37: Spiral - Project complexity first criteria ............................................................................. 147 Figure 96: Spiral – Project complexity second criteria ....................................................................... 148 Figure 97: Spiral – Projects requirements defined at the beginning of the project ............................. 149 Figure 98: Spiral – Rate of change in projects by request................................................................... 149 Figure 99: Spiral – Level of quality expected in the projects.............................................................. 150 Table 39: Spiral - Investigated project status and duration ................................................................ 151 Table 40: Spiral – How critical is the failure of project to the organisation....................................... 151 Figure 100: Spiral – Projects budget ................................................................................................... 152 Figure 101: Spiral – Project team’s experience................................................................................... 152 Figure 102: Spiral – Team empowerment ........................................................................................... 153 Figure 103: Spiral – Frequent team meeting (fortnightly)................................................................... 154 Figure 104: Spiral – Common communication methods among team members................................. 154 Figure 105: Spiral – Common communication methods between team members and managers ....... 155 Figure 106: Spiral – Project Management/Leadership main roles and experience ............................. 156 Figure 107: Spiral – Project Management/Leadership experience...................................................... 157 Figure 108: Spiral – Techniques used in projects................................................................................ 157 Figure 109: Spiral – Advantages/benefits of the used lifecycles......................................................... 158 Figure 110: Spiral – Disadvantages/limitations of the used lifecycles................................................ 159 Figure 1. Waterfall model with single-level feedback paths ............................................................... 189 Figure 2. Waterfall model.................................................................................................................... 190 Figure 3. Spiral model for Software Development............................................................................. 190 Figure 4. Spiral model- Adapted from B.W.Boehm........................................................................... 191 Figure 5. Incremental for Software Development .............................................................................. 192 Figure 6. Extreme Programming (XP) lifecycle................................................................................. 193 Figure 7. Scrum process ..................................................................................................................... 193 Figure 8. DSDM ................................................................................................................................. 194 Figure 9. Adaptive Software Development ........................................................................................ 194 Figure 10: Crystal family of methodologies....................................................................................... 194 Table 2: Organisation’s nature – question 7........................................................................................ 195 Figure 12: Organisation Characterisation (Public sector) ................................................................... 195 Figure 13: Organisation Characterisation (Private sector) .................................................................. 196 Figure 14: Organisation’s Leadership (Public sector) ......................................................................... 196 Table 3: Organisation’s Leadership – question 8 ............................................................................... 197


- vii -

Figure 15: Organisation’s Leadership (Private sector)........................................................................ 197 Table 4: Organisation’s Management – question 9 ............................................................................ 197 Figure 16: Organisation’s Management quality (Public sector).......................................................... 198 Figure 17: Organisation’s Management quality (Private sector)......................................................... 198 Table 5: Organisation’s Management style/qualities – question 10................................................... 199 Figure 18: Organisation’s Management styles (Public sector)............................................................ 199 Figure 19: Organisation’s Management styles (Private sector)........................................................... 200 Figure 20: Organisation’s Emphasis (Public sector) ........................................................................... 200 Figure 21: Organisation’s Emphasis (Private sector) .......................................................................... 201 Table 6: Organisation’s Emphasis – question 11 ............................................................................... 201 Table 7: Organisation’s Definition of success – question 12 ............................................................. 201 Figure 22: Organisation’s Definition of success (Public sector) ......................................................... 202 Figure 23: Organisation’s Definition of success (Private sector) ........................................................ 202 Table 16: Waterfall – Question 50 responses..................................................................................... 204 Table 22: Agile – Question 50 responses ........................................................................................... 206 Table 29: Incremental – Question 50 responses ................................................................................. 207 Table 35: Iterative – Question 50 responses....................................................................................... 207 Table 41: Spiral – Question 50 responses .......................................................................................... 207 Table 12: Questions – rate of change in project by requests .............................................................. 210


- 8 -

Acknowledgements

I would like to express my deep and sincere gratitude to my

supervisor Dr. Clive Boughton, Adjunct Associate Professor at

Australian National University. His encouraging, patience,

motivation, enthusiasm and personal guidance have provided a

good basis to conduct this research project.

I would like also to thank all the participants, who gave their

time to complete the questionnaire and provided supportive

comments. Also I would like to thank all people who spent time

to provide me with feedback on the designed questionnaire.

Finally, I owe my loving thanks to my family for their

encouragement, support, patience and understanding, who

supported me all the way of my master.

Many thanks to all of you


- 9 -

CHAPTER 1

Introduction Software Development Methodologies (SDM) in industry.

Nowadays our daily life becomes more and more reliant on technology, especially software

products. Software engages much equipment and facilities that we use every day. Our daily

transactions are no longer manually processed. The number of transactions processed by

softwares has reached to the hundreds of millions a day.

For example: “Every day 268 million people use Google to search for something. The query goes

in, the company's software delivers back the most relevant links” [41].

Software is growing more and more in size, cost and complexity. Even though software has been

written for more than 40 years, still customers/stakeholders complain about the produced

softwares/applications as hard/difficult to use, requirements are not met, project delivered late or

over budget. On the production side, software development teams complain about project

requirements as being difficult to prioritise, requirements changes are difficult to control, the

development process is difficult to control.

Software engineering is the practice of using selected process techniques to improve the quality

of a software development effort. This is based on the assumption, that a systematic approach to

software development, results in more control of the development process, more control of

requirements changes, less product defects, deliver product on time, deliver high quality product

and eventually get customer satisfaction.

If we think about the following questions, they may help to understand the values behind the

using of software development methodologies in our industry.

In terms of software bugs/errors, how much does it cost Australian economy?

In Australia I couldn’t find any empirical data/figure about the cost impact of software bugs on

our economy. But, in U.S.A. alone, $59.5 billion a year is wasted on software bugs, according to

[22].

Delays in software delivery, how much does it cost Australian economy?


- 10 -

Maintenance of software code or undocumented softwares/applications, how much does it cost

Australian economy?

How we could avoid or reduce software failure?

These are a very straightforward questions and the answers on these questions, involves many

factors, including the right using of software development methodologies.

The collected documents of policies, processes and procedures etc used by a development team

or organisation to practice software engineering is called software development methodology

SDM) or software development lifecycle (SDLC) [42].

A methodology is composed of one of the software development lifecycles used in conjunction

with one or more techniques.

This report consists of four chapters. All chapters are related to the research project of the

investigated software development methodologies. This introductory chapter is an overview of

software development methodologies and its effects, also project purpose, scop and objective.

Chapter two briefly describe the background of software development lifecycles. Chapter three

describe the most common research methods and the reasons behind the selected method for my

research project, at this chapter also we look at survey objectives, questionnaire design as well as

risk management and the alternative solution. Finally chapter four will explore the investigation

of the software development methodologies results, including the analysis, conclusion and future

study.

Finally, due to the size of the data received from questionnaire responses, and in order to gain the

best results, I used all the data that fit with the analysis. But, to minimise the size without losing

the value of the report, I have put some of the analysed results (especially the graphs and tables)

in the appendixes. Most of the analysed data are self explanatory, and I tried to avoid repetition.

Purpose and Scope Investigation of different types of software development methodologies in industry will be

conducted. The usage of these methods in organisation/industry and what can be learned from

their usage, by conducting a survey, and examine these methods/practices and what makes these

practices beneficial and why they are less suitable for certain projects or situations.


- 11 -

This project will introduce software development methodologies. The information will help the

development team/organisation to recognise which methodologies may be best suited for use in

various situations. The target audience is all personnel involved in software development within

IT industry. It will cover the methodologies that are most applicable for small and big projects

that can be implemented/accomplished by one or more software development methodologies.

The intended readers should consider government software standards in their use of

methodologies.

Objective and mission implementation The objective of this project is to practically evaluate how Software Development Methodologies

(SDM) are actually used in industry and what the industry says on their practices. The usage of

these methods (lifecycles and techniques) in organisation/industry and what can be learned from

their usage and what makes these practices beneficial and why they are less suitable for certain

projects or situations.

In order to achieve project objective, initially, a research needs to be conducted, the data can be

secure by conducting an investigating on individual projects and the usage of development

methodologies in industry. Next, the results of the conducted research have to be analysed,

resultant in recommendations (to help improve current development process) and in suggestion

of future studies. The recommendations and suggestion should be based on the conducted

research results, evaluation and performed literature. Finally, I shall produce usable table of

software development lifecycles (see Appendix A).


- 12 -

Terminology

Software Development Methodology SDM

System Development Lifecycle SDLC

Extreme Programming XP

Dynamic Systems Development Method DSDM

Rapid Application Development RAD

Adaptive Software Development ASD

Internet Speed Development ISP

Lean Software Development LSD

Joint Application Development JAD

Microsoft MS

Australian National University ANU

Information Technology IT

Rational Unified Process RUP

Australian Bureau of Statistics ABS

Object-Oriented OO

Procedural Language/Structured Query Language PL/SQL

Statistical Analysis System SAS


- 13 -

CHAPTER 2

Background of Software Development Lifecycles

For many decades, people from both academic and industry backgrounds have tryed to find

methodologies or processes which are repeatable (frequency of use), predictable, effective and

flexible, that increase productivity and improve quality.

As demands on using software are increasing, accordingly software products are rapidly

changing and growing in size, complexity and cost in both time and money, on the other side,

this increases the pressure on the development team, therefore, software development

methodologies are getting more complex and difficult to apply, if also take into account the

affect of the factors on the development process. Refer to figure 10 crystal family of

methodologies, also figures 6 & 7 for XP and Scrum respectively. In order for a development

teams to use any of these methods without failure, they must have high experience and the team

familiar with the development process.

“One of the main reasons for using a software development methodology is to support and

enhance communication among stakeholders during all of the development phases” [32].

In this chapter I will go through the background of the methodologies that are used in the survey.

Waterfall

The classic waterfall model is a popular version of the SDLC for software engineering, which

was introduced in the 1970s by Win Royce at Lockheed. It is so named because it can be

represented or graphically modelled as a cascade from establishing requirements, to design

creation, to program implementation, to system test, to release to customer, as shown in Figure 1.

It was a great step forward in software development as an engineering discipline. The figure also

depicts the single-level feedback paths that were not part of the original model but that have been

added to all subsequent improvements of the model. The original waterfall model had little or no

feedback between stages; it describes a development method that is linear and sequential.

Waterfall development has distinct gals for each phase of development, just as water does not


- 14 -

reverse or flow uphill. Imagine a waterfall on the cliff of a steep mountain. Once the water has

flowed over the edge of the cliff and has begun its journey down the side of the mountain, it

can’t turn back. It is the same with waterfall development. Once a phase of development is

completed, the development proceeds to the next phase and there is no turning back.

This method might work satisfactorily if design requirements could be perfectly addressed before

flowing down to design creation, and if the design were perfect when program implementation

began, and if the code were perfect before testing began, and if testing guaranteed that no bugs

remained in the code before the users applied it, and of course if the users never changed their

minds about requirements. Unfortunately, from experience, none of these things is ever true [43].

In figure 2 [42] the approach is used in high risk projects, particularly large defence contracts.

The problems in waterfall do not arise from "immature engineering practices, particularly in

requirements analysis and requirements management." Studies of the failure rate of the DOD-

STD-2167 [44] specification, which enforced waterfall, have shown that the more closely a

project follows its process, specifically in up front requirements gathering, the more likely the

project is to release features that are not used in their current form.

In terms software sizes and complexity and millions lines of code in current systems, it is hardly

for waterfall handle such projects.

The advantage of waterfall devolvement method is that, it allows for departmentalization and

managerial control. A schedule with deadlines for each stage of the development can be set and a

product can proceed through the development process, theoretically, to be delivered on time. In

more simplicity, development moves from concept, through design, implementation, testing,

installation and ends up in production and maintenance, each phase of development proceeds in

strict order without any overlapping or iterative steps.

The disadvantage of waterfall development method/model is that it does not allow for much

reflection or revision. Once an application/system is in the testing stage, it is very difficult to go

back and change something that was not well-thought out in the concept stage. [43]

For Classical Waterfall model refer to Appendix F.

For Waterfall model with single-level feedback paths refer to figure 1, Appendix F


- 15 -

For adaptive Waterfall model refer to figure 2, Appendix F

Spiral While the waterfall methodology offers an orderly structure for software development, demands

for reduced time-to-market make its series steps inappropriate. The next evolutionary step from

the waterfall is where the various steps are staged for multiple deliveries or handoffs, as shown

in figure 3.

The ultimate evolution from the waterfall is the spiral, taking advantage of the fact that

development projects work best when they are both incremental and iterative, where the

development team is able to start with a small team and benefit/learn from progressive trial and

error along the way.

The spiral methodology reflects the relationship of tasks with rapid prototyping, increased

parallelism and concurrency in design and builds activities. The spiral method should still be

planned methodically, with tasks and deliverables identified for each step in the spiral [42].

The spiral model, developed by Dr. Barry Boehm at TRW, is an enhancement of the

waterfall/rapid prototype model, with risk analysis preceding each phase of the cascade. You can

imagine the rapid prototyping model drawn in the form of a spiral, as shown in Figure 4 [2]. This

model has been successfully used for the internal development of large systems and is especially

useful when software reuse is a goal and when specific quality objectives can be incorporated. It

does depend on being able to accurately assess risks during development. This depends on

controlling all factors and eliminating or at least minimizing external influences. Like the other

extensions of and improvements to the waterfall model, it adds feedback to earlier stages. This

model has seen service in the development of major programming projects over a number of

years, and is well documented in publications by Boehm and others [43].

For Spiral model for Software Development refer to figure 3, Appendix F For Spiral model- Adapted from B.W.Boehm refer to figure 4, Appendix F


- 16 -

Incremental The incremental recognizes that software development steps are not discrete. Instead, it consists

of builds of prototypes, so Build 1 (a prototype) is improved and functionality/features are added

until it becomes Build 2, which becomes Build 3, and so on. These builds are not the versions

released of the product to the customer, but are merely staged compilations of the developing

system at a new level of functionality or completeness. As a major system nears completion, the

project manager may schedule a new build every week. Surprisingly, the team who does not

have their module ready for the build or whose module causes the regression testing to fail, As

Figure 5 shows, the incremental model is a variant of the waterfall and rapid prototyping models.

It is intended to deliver an operational system with high quality that meets customer

requirements at each build, but it does not yet complete the functional specification.

The advantages of the incremental are:

It is flexible enough to respond to critical specification changes as development progresses.

Also the analysts and developers can tackle smaller chunks of complexity. Users and developers

both learn from a new system’s development process, and any model that allows them to

incorporate this learning into the product is advantageous.

The disadvantage

Learning exceeds productivity and the development project becomes a research project

exceeding time and budget or even worse, it never delivers the product at all. However, learning

need not exceed productivity if the development team remains aware of the risks and focused on

customer requirements. As stated by [43].

For Incremental lifecycle refer to figure 5, Appendix F


- 17 -

Agile development methodologies

Extreme Programming (XP)

Extreme programming is one of the agile development methodologies. It is the development of

the incremental model that puts the client in the right path. Each feature or feature set of the final

product envisioned by the client and the development team is individually scoped for cost and

development time (project resources). The client then selects features that will be included in the

next build (again, a build is an operational system at some level of functionally) based on a cost

benefit analysis.

Extreme Programming (XP) is an agile software development method that was created by Kent

Beck and was first used in 1996 in a project at Chrysler [43]

Figure 6 below (source [5] ), shows the lifecycle of the XP process. The XP lifecycle consists of

six phases:

1. Exploration phase.

2. Planning phase

3. Iterations to release phase

4. Product-ionizing phase

5. Maintenance phase

6. Death phase.

For Extreme Programming (XP) lifecycle model refer to figure 6, Appendix F During the exploration phase, the customer writes his requirements on so-called story cards

(cards which contain a feature to be implemented) and the developers work on technologies to be

used in the project. Based on this preparation, a prototype of the system might be built.

Within the planning phase, also known as the “planning game”, the developers estimate for each

card how long it would take to implement this card and based on this estimations, customer and

developers decide together about the prioritization of each card. According to this prioritization,

a release plan/schedule is finally set up which says which feature will be implemented in each

release.


- 18 -

In the iterations to release phase the prioritized story cards are implemented in iterations of one

to four weeks. In these iterations, the design as well as the coding is done, but before any line of

code is written, first a unit test to test these lines has to be developed by the programmers.

Finally, as soon as the developed features are tested by the developers (probably by automated

unit tests), they are given to the customer and thereby, the next phase is entered.

The product- ionizing phase mainly concerns performance and system testing. In this phase, the

customer performs functional tests and validates if the product works as in-tended. If new

requirements are elicited, they are either included directly or a new story card is created which

will be considered in a following release planning.

Throughout the maintenance phase, the customer is supported by (probably new) team members

whose task is to ensure that certain customer requests as e.g. improvement suggestions are

considered.

In the death phase, the XP project is either successfully finished or finished without the expected

results, e.g. if the budget is overrun. In both cases, no changes to the sys-tem are made anymore.

The only action that might still happen in this phase is writing documentation [5].

In addition to the phases as presented above, the typical roles in the XP process are illustrated in

the analysis below, as stated by [5] and [6]

Scrum

The Scrum methodology was invented in 1993 and developed in 1996 by Jeff Sutherland and

Ken Schwaber [7].

It is worth to mention that, XP and Scrum are an iterative software development method.

Scrum process are divided into 3 phases:

1. Pregame,

2. Development,

3. Postgame (figure 7 – source [5]) which we will present in the following according to [5].

The pregame phase is sub-divided into planning and architecture/high level design. During the

planning, a so-called product backlog list (a requirements specification) is written, the

requirements are prioritized and the development effort is estimated. Moreover, the project

resources (time, budget and staff) are fixed. Throughout architecture/high level design, the high


- 19 -

level design based on the product backlog is defined and reviewed in a meeting where

implementation details are decided.

Secondly, the development phase consists of so-called sprints (iterative software development

cycles that last from one to four weeks) which result in a release. The sprints are based on a

sprint backlog list (a document that contains the features to be implemented in the sprint). Each

sprint includes activities of traditional software development approaches (e.g. requirements

engineering, design, coding etc.) and can be performed by several teams.

Finally, the postgame phase is characterized by documentation, integration testing and system

testing. The postgame phase ends with the final release of the software.

Typical Scrum roles are illustrated with the analysis below as indicated by [5] and [8].

For Scrum model process refer to figure 7, Appendix F

Dynamic Systems Development Method (DSDM)

DSDM has its origin in 1994 and was established by a group of 16 people, the DSDM

consortium [9]. It is a Rapid Application Development (RAD) framework [10] with the basic idea

of fixing the project resources before fixing the functionality [5]. In figure 8 (source [5]), the

DSDM process is divided into five phases:

1. Feasibility study

2. Business study

3. Functional model iteration

4. Design and build iteration and

5. Implementation phase, which are explained below according to [5].

For DSDM model refer to figure 8, Appendix F

1. Feasibility study The DSDM process starts with the feasibility study. The feasibility study is utilized to evaluate

whether the planned project is technically feasible and whether to use or not use DSDM. The

feasibility study results in the feasibility report (describes the feasibility of the project) and the


- 20 -

development outline (general outline of the system to be developed). If the technologies to be

used are not well known, a prototype can be built within the feasibility study.

2. Business study

During the business study, the technological and business requirements are specified and

prioritized with the help of the customer. The business study results in the business area

definition (process descriptions with the help of e.g. entity-relationship diagrams and

specification of user classes to be involved), the system architecture definition (a first

architecture draft), and the prototyping plan (a plan including the prototyping strategy and

configuration management).

3. Functional model iteration

After the business study, in the functional model iteration, several iterations are used to find a

functional model for the system that is to be developed. Finding this model also includes the

development of prototypes and depending on the quality of these prototypes, it might even occur

that parts of these prototypes are later integrated in the final system. The functional model

iteration results in a functional model, including prototype code and analysis models. Besides,

the functional model iteration produces a list of prioritized functions, functional prototyping

review documents including user comments, a list of non-functional requirements to be included,

and a document about risk analysis of further development.

4. Design and build iteration

In the design and build iteration phase the system is completed based on the most important

requirements. This completion is also done by several iterations. As well as in the preceding

phase, prototypes are built, tested, and reviewed by the users.

5. Implementation

The implementation phase (which is also done in iterations) is utilized to transfer the system into

its real application field at the customer. If it was not possible to complete the system in the

given time, the DSDM process might be run again in order to include remaining features.

The implementation phase results in two things, first, a user manual which explains to the users

how to use the system, and second, in a project review document in which the project’s outcome

is specified.

DSDM roles as stated by [9] and [5] are:


- 21 -

• Executive sponsor (customer representative with financial authority),

• Visionary (customer representative with high knowledge of the business do-main),

• Ambassador user (spreads information about the project progress and user in-formation ),

• Project Manager,

• Technical coordinator (responsibility for the system architecture, technical quality, and

configuration management),

• Team Leader (ensures team work),

• Developer (transfers requirements into code),

• Tester (responsibility for the non-user testing),

• Scribe (makes notes of requirements in meetings and workshops), and

• Facilitator (manages workshops).

Adaptive Software Development

Adaptive Software Development (ASD) was developed by Jim Highsmith and was first

published in 1997 in [11]. The methodology is focused on iterations and constant prototyping and

is basically supposed to be used for the development of large software systems [5]. The ASD

process is divided into three phases:

1. Speculate

2. Collaborate, and

3. Learn

The explanation of the (ASD) below as shown in figure 9 (source [11]) is according to [11], [5].

For Adaptive Software Development model refer to figure 9, Appendix F During the speculate phase, the objectives, vision, goals, and requirements of the sys-tem to be

developed are specified. These artefacts are also called the mission of the project and they are

identified with Joint Application Development (JAD) sessions, i.e. workshops with customer

representatives and developers. ASD uses the terms speculate due to Highsmith’s opinion that in

large environments outcomes are unpredictable.

Secondly, the collaborate phase mainly deals with the iterative development of product

components. ASD rather emphasizes the quality of the components which is reached by

teamwork. At the end of each iteration, quality reviews are held with the help of the customer.


- 22 -

Finally, the learn phase concerns final quality assurance and lessons learned sessions. The

lessons learned sessions are important for future projects because the employees shall avoid

mistakes they made before.

ASD roles as stated by [5] are:

• Executive sponsor (has the whole project responsibility)

• Facilitator (organizes the JAD sessions)

• Scribe (makes notes during the JAD sessions)

• Project Manager

• Customer, and

• Developer

Crystal

The Crystal Family of Methodologies (Crystal) was developed by Alistair Cockburn [12]. Crystal

is based on several software development methods in order to choose the most appropriate

technique for a project [5] and is a typical people-centric methodology since it focuses on

improving the human power [12]. The methods are shown in figure 10 (source [12]), where they

are categorized by the size of the project by the number of participating employees (x-axis) and

the criticality of the system (y-axis).

For Crystal family of methodologies model refer to figure 10, Appendix F


- 23 -

CHAPTER 3

Research Methods

Introduction

As a computer systems engineer, working in the IT industry with over 5 years experience and

currently specialising in studying software engineering at ANU, as part of the masters degree

requirement to undertake a research project, I have decided to conduct a survey as part of the

research project on software development methodologies in industry, in order to find out

something (more than hearsay) about the sorts of development methods and project life cycles

now being applied, and also some of the attitudes/conditions that exist surrounding organisations

and their staff. In chapter 2 I’ve identified the most common software development lifecycles. In

this chapter, I will illustrate the appropriate empirical research method used in the research

project, by describing the most common types of research methods, and will explain the reason

behind the decision of using survey method (questionnaire) as the best research method for my

project.

Description of the most common research methods

“There are three major different types of investigations (Strategies) that may be carried out” [13]

1. Survey

2. Case study

3. Experiment

Every one of these types can be use for different purposes [13]. Now I will briefly explain the

three types (as stated by [13]) and the reason of my chosen one for my research.

Survey type: A survey is often an investigation performed in retrospect, when, for example, a

tool or a technique, has been use for a while. The primary means of gathering qualitative data are

interviews or questionnaire. These are done through taking a sample which is representative from

the population to be studied. The results from the survey are then analysed to derive descriptive

and explanatory conclusions. They are then generalized to the population from which the sample

was taken [13].


- 24 -

Case study: Case studies are used for monitoring projects, activities or assignments. Data is

collected for a specific purpose throughout the study. Based on the data collection, statistical

analyses can be carried out. The case study is normally aimed at tracking a specific attribute or

establishing relationships between different attributes. The level of control is lower in a case

study than in an experiment. A case study is an observational study while the experiment is a

controlled study. A case study may, for example, be aimed at building a model predict the

number of faults in testing. Multivariate statistical analysis is often applied in this type of studies.

The analysis methods include linear regression and principle component analysis [13].

Experiment: experiments are normally done in a libratory environment, which provides a high

level of control. When experimenting, subjects are assigned to different treatments at random.

The objective is to manipulate one or more variables and control all other variables at fixed

levels. The effect of manipulation is measured, and based on this a statistical analysis can be

performed. In some cases it may be possible to use true experimentation, we may have to use

quasi-experiments. The letter term is often used when it is impossible to perform random

assignment of the subjects to the different treatments. An example of an experiment in software

engineering is to compare two different methods for inspections. For this type of studies methods

for statistical inference are applied with the purpose of showing with statistical significance that

one method is better than the other [13].

Selected research type

On the light of the previous explanation, and base of the most suitable and appropriate research

method (for the reasons stated briefly below) for my research project purposes, I have decided to

conduct a questionnaire survey.

1. First reason, my research project is based on the practicing and usage of the software

development lifecycles and techniques in industry. One of the research sub-objective is to

collect data from an experience people whom practicing and have knowledge and solid

experience in software development methodologies in our industry.

2. Second reason, conducting a questionnaire survey by sending emails to intended

participants is quicker way and cheap than other methods. In fact I have confronted a

problem in collecting email addresses (for intended participants) from companies and


- 25 -

organisations, in order to circulate emails to invitees to participate in the survey. My

project supervisor Dr Clive Boughton has encouraged me to proceed, and he took the

responsibility of this side of the research.

3. Third reason. “The primary means of gathering qualitative or quantitative data are

interviews or questionnaire” [13]. Based on that, collecting data by conducting a survey is

more beneficial and more suitable to my research objective.

Survey (Questionnaire)

After I have illustrated the most common research methods, and the reasons behind my decision

to conduct a survey as the research method (questionnaire- appendix B) for the project, it is

important to set up the survey objectives before designing and constructed the questionnaire,

otherwise the collected data could be ineffective (or useless) and not covering all aspects of the

survey objectives.

In fact, I have confronted a lot of difficulties at the beginning, the questionnaire become a project

by it self, it took me about or even more than five months working on this survey/questionnaire.

The difficulties were varies at all stages, questionnaire objectives, design a meaningful questions

and structure and grouping, time consumption (stress me out) and email addresses of participants

(invitees’ email) etc. Every issue/problem have been dealt with and solved with the guidance of

Dr Clive Boughton (project supervisor).


- 26 -

Questionnaire objectives

The questionnaire objectives are basically driven from the project objective, but not limited to

that. The main aim of the project is to investigate different types of software development

methodologies used in the software industry as mentioned above (see project objective).

Survey/questionnaire objectives as follow:

1. Collect data from participants about the usage of software development methodologies in

the organisation.

2. Collect data from participants about organisation size

3. Collect data from participants about the techniques that used with the development

lifecycle in the projects

4. Collect data from participants about organisations culture and work environment

5. Investigate individual project, by collecting the following:

6. Collect data from participants about project status and duration

7. Collect data from participants about project complexity

8. Collect data from participants about the used development lifecycle and techniques.

9. Collect data from participants about rate of change in project

10. Collect data from participants about project constraints

11. Collect data from participants about project criticality

12. Collect data from participants about team factors in the project

13. Collect data from participants about management/leadership experience

14. Finally collect general information based on participants experience including advantages

and disadvantages of the used software development lifecycles.

Questionnaire design The questionnaire design (appendix C) will be based on the survey/questionnaire objectives

mentioned above. There are many aspects that should be considered. First, define participants


- 27 -

and their roles. The questionnaire is intended for professionals who typically work within a

software team, such as team leader, project manager, business analyst, software architect and

designer.

Second, questionnaire completion time should not exceed 30 minutes, participants time must be

valued. Therefore, I have decided to construct the questionnaire in three sections:

1. First section will contain research questions that will address the usage of software

development lifecycles, techniques and organisational culture.

2. Second section will investigate individual projects and will include, duration of project,

complexity, rate of change in project, project constraints including quality, project

criticality, development team factors, management/leadership experience, communication

and the lifecycles and techniques used in project.

3. Third section will gather general information, advantages and disadvantages and feedback

on the used methodologies. For questionnaire design see appendix C.

After the design of the questionnaire, the questionnaire (see appendix B) has been reviewed by

many people from software industry for the following reasons (see feedback sheet appendix E):

1. To ensure that the questions are consistent, understandable and easy to answer

2. To get corrections and suggestion in regard to English language.

3. To get feedback about questionnaire completion time (in between 15 – 20 minutes) should

not exceed 30 minutes.

4. To ensure that the naming convention and terminology used are correct and

understandable.

Conducting the Survey (questionnaire)

After completing the design of the questionnaire, we moved to the implement it. The

questionnaire was sent to participants by Email. An Email template (appendix D) was completed

to be used as a standard, and to be sent to all intended participants/invitees to participate in the

survey. An MS access database was designed to store the returned data from participants. I used

two tools for data storage, MS access database and MS Excel. Initially all the data was stored in

access database, then data was transferred to excel for further analysis and results. The


- 28 -

advantages of storing data in Access database are, more control over data, easy and comfortable

to handle data, easy to use query and reports to retrieve and view data and no database server

required to be installed.

Responses expectation and alternatives

The survey was initially sent to ~110 invitees (participants) by Dr Clive (project supervisor) to

organisations from both public and private sectors, and they were asked to send on the

questionnaire to people they knew who would be interested in participating in this survey. Some

invitees/participants did do that, and some of the 29 responses include that secondary

distribution. Therefore, based on what has been mentioned above, a further 20 invitees received

the questionnaire, making a total of ~130. This means that the response rate reached 22.3%. Our

response expectation for the survey was 10-20%, and the actual response rate is 22.3%, which is

quite good and met our expectation. Some of the reasons behind not perhaps achieving a higher

response rate were: some people they move position in different organisation/company; some

emails bounced and did not reach their destination; finally, some people although willing were

too busy to participate in the survey.

Some feedback, one person pointed out that they felt the survey was going to be too time-

consuming, and ultimately provided no response.

One person provided some comment on the difficulty that they had filling out the questionnaire.

Risk management, in order to continue with my project, an alternative solution (risk plan)

had been prepared in case of not receiving enough responses on the survey or in case of any

other serious issues/problems. A case study would take the place of the survey on software

development methodologies using different resources, including interview questions. The plan

was to conduct at least 6 interviews with people from the software development industry. I

understand that it is very hard to convince people to participate in interviews, and also

conducting a case study by interviews takes more time than a questionnaire.

There were no incidents recorded that could cause failure to either the questionnaire or the

project. Therefore, the alternative plan was never used.


- 29 -

CHAPTER 4

Results Analysis, Conclusion and Future study In this chapter I will illustrate and discuss the results of the questionnaire. As mentioned above

the collected data have been stored in an MS Access database, the data were retrieved and

grouped according to the targeting criteria, e.g. at the first section (Lifecycles, techniques and

organisation culture) data grouped by participant sector type, in second section (investigation of

individual projects) data grouped by lifecycle type. The results of the statistical analysis will be

presented in different forms, follow by conclusions, recommendations and future study.

The expected results of the investigated of projects by used lifecycles, will be presented at the

end of the investigation section for the particular lifecycle in connection with appendixes. (E.g.

the projects used waterfall lifecycle will be investigated, and results will be presented under

Waterfall lifecycle section, as well as in appendixes as pointed in the related places).

I will commence the reporting of results with some statistical figures, and then will proceed with

the analysis of the data. The analysis will be based on the questionnaire objectives and performed

literature where ever data was found.

Statistical figures

The results show that 83% of the involved organisations used one or more of the development

lifecycles listed in the questionnaire. 17% have used other lifecycles. 34.5% of participants were

from public sector and 65.5% were from private sector.

Table 1 illustrates the number of organisations that used particular lifecycles and number of

projects that used a particular lifecycle. Note some organisations have used more than one

lifecycle in their projects.


- 30 -

Lifecycles No of Organisations No of Projects used

Agile 13 55

Incremental 7 17

Iterative 10 64

Spiral 5 24

Waterfall 15 66

Other 5 >23

No lifecycle 1 8

Table 1: SDLC by number of organisation and projects

The statistical figures stated above will be used in the coming analysis.

The figures in Table 1 show that, the most commonly used lifecycles by participating

organisations are: Waterfall, used in 66 projects by 15 organisations, then Iterative used in 64

projects by 10 organisations, Agile used in 55 projects by 13 organisations, Spiral used in 24

projects by 5 organisations, Incremental used in 17 projects by 7 organisations. The participating

organisations used other lifecycles as follow:

Two organisations used Rational Unified Process (RUP), one organisation developed its own

methodology in-house, which is driven from other software development lifecycles beside 4

other lifecycles used in the questionnaire (as stated by participant), the second organisation used

what they call the “personal software process” and “team software process” beside 2 other

lifecycles used in the questionnaire.

Analysis

In order for readers of this report to gain a clear picture and understand the analysis and the

results, I suggest that they look at the questionnaire design (see appendix B).

Proceeding with this chapter, the analysis of the data will be based on questionnaire design

sections (questionnaire structure). The questions in the questionnaire have been designed and


- 31 -

grouped according to the survey objectives, this way will made it easier to correlate responses to

particular sets of questions.

In many places in the report I will used organisation size, therefore, it must be made clear to

readers of the exact definition of small size, medium size and large size (in terms of organisation

size). All the way through this report I will use the definition of Australian Bureau of Statistics

(ABS) in defining small, medium and large businesses. According to [26]:

• Small business is defined as a business employing less than 20 people

• Medium businesses - employ 20 or more people, but less than 200 people • Large businesses - employ 200 or more people

Eventually, the purpose of this analysis is to gather data from the investigated projects and

provide significant statistical analysis that could add value into this field, also to advice software

development practitioner of the most suitable development methodology for a particular project

and situation, as well as the advantages and disadvantages of using such methodology. “We need

some better advice on how and when to use methodologies”, as stated by [23].

In order to make the research more valuable, I must compare the results of the analysis with

other surveys, but, unfortunately I couldn’t find any similar project/survey to compare them.

Therefore, I will use published literature (searched references) with the findings of this analysis,

where applicable.

The project objective started with one paragraph, then selected the right research method

(survey), then set up research objectives based on the project objective, followed by the

questionnaire design (based on the research objectives), then the construction of the

questionnaire was based on the questionnaire design which was used as a roadmap to the

analysis. Hopefully this systematic approach will gain the maximum benefit from the

questionnaire responses and the large amount of collected data.

Organisational culture, Lifecycles and Techniques

The first section in the questionnaire design deal with the lifecycles, techniques and the

surrounding culture (work environment) within the organisation.


- 32 -

The questions that cover these criteria are from questions 1-12. These questions do not

investigate projects, rather they target SDLC usage in organisations and the culture/environment

surrounding them.

SDLC usage by number of organisations and sizes

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Agile Incremental Iterative Spiral Waterfall Other

Software Development Lifecycles (SDLC)

Num

ber o

f Org

anis

atio

ns

4 - 10000

1 - 300

4 - 300

6 - 10000

4 - 10000

24 - 300

Figure 11: Number of SDLC with organisation size

Figure 11 illustrates the usage of the software development lifecycles by number of organisations

and sizes (number of staff). The data in this graph was obtained from participants’ responses to

questions 2 and 3 in the questionnaire. The y-axis shows the number of organisations and the x-

axis shows the development lifecycle. The numbers on the top of the bars represent

organisations size (by staff number). E.g. Spiral used in organisations/companies with various

sizes, range from 1 person to a staff of 300. The result is for both sectors

The result shows that, the usage of the common lifecycles used in industry are not

restricted/determined by organisation size.

Continuing with the analysis of questions 7-12, the questions have been set to collect data from

participants about organisational culture.


- 33 -

“Most organisational scholars and observers now recognize that organisational culture has a

powerful effect on the performance and long-term effectiveness of organizations. Empirical

research has produced an impressive array of findings demonstrating the importance of culture to

enhancing organizational performance” [16].

In general, the work environment has an impact and influence on the methods in use. Also, the

work environment has an impact on the project development progress, quality, performance and

outcomes.

“It depends on the environment. In the enterprise systems environment (by which I mean the

environment where systems are built around an existing hardware and software infrastructure,

and aimed largely at automating or improving an existing business process) any predictive

methodology (such as waterfall) tends to be brittle, expensive and largely ineffective. If however

the organisation demands upfront costing it is hard to avoid a waterfall model.

Agile should also be used with caution. It tends to emphasise the programming team and de-

emphasise the architecture of the system. This fails to recognise the role that architecture has in

change management” according to project 1 (table 22: Agile-Question 50 responses) Appendix H

“Agile probably wouldn't work so well in an environment where the team wasn't experienced /

technically capable and the management wasn't willing to support the methodology. The team

needs to be willing to take on the challenge of learning to follow agile but in doing so they need

to have enough technical skills to confidently produce useful deliverables every iteration with

limited up-front analysis. Additionally, if management doesn't support the methodology then you

risk being stranded/isolated if an issue arises that needs to be” according to project 4 (table 22:

Agile-Question 50 responses) Appendix H

“In fact, it is difficult to name even a single highly successful company, one that is a recognized

leader in its industry that does not have a distinctive, readily identifiable organisational culture”

[9]. “Organisational culture has a powerful effect on the performance and long-term effectiveness

of organisations” [16].

The important question that shall be asked,

What are the environmental factors that best suit particular development lifecycles? Figures 12-23 shows results of the collected data from questions 7-12 (the questions illustrated in

Tables 2-7), the data are grouped by public and private sectors. The six tables mentioned above


- 34 -

consist of the statements (description) and indicators (letters A, B, C, D) to connect the graphs

mentioned above with the corresponding statements in the questions.

The coloured bars in the six figures represent participants’ responses on every statement in the

questions, the responses on each statement will be one of; strongly agree, agree, neutral, disagree

or strongly disagree.

For example, figure 7 shows results of the collected data from question 7, the letters A, B, C and

D correspond to the statements in question 7.

Note that the results of these questions (7-12) are presented in figures (12-23) are based on

participant’s sector.

For figures 12-23 and tables 2-7 refer to Appendix G.

Figure 12 show the results for organisation characteristics, refer to table 2 for statements details.

“One of the approaches to enhance organisational performance is reengineering, the attempt to

completely redesign the processes and procedures in an organisation” [16].

“Organisational culture has a powerful effect on the performance and long-term effectiveness of

organisations”[16].

Results of figure 12 show that, participants agree partially on statements A and C.

Figure 13 show the results for organisation characteristics, refer to table 2 for statements details.

“Some organisations are viewed as effective if they have harmonious internal characteristics”

(16).

The results in figure 13 show positive characteristics, participants agreed on four statements (A, B, C, D).

Figures 14 and 15 shows the results for organisation’s leadership, question 8 (table 3).

Figure 14 show participants agreed on two statements A and C. refer to question 8 for full explanation.

Figure 15 shows participants agreed on three statements A, B and C.

No evidence found to support these criteria with particular lifecycle. Figures 16 and 17 shows the results for organisation’s management quality, question 9 (table 5)


- 35 -

Figure 16 shows that, participants agreed only on one statement (D), which mean management

quality, need to be reviewed for these organisations.

Figure 17 shows that, participants agreed on three statements (A, C and D), participants satisfaction of management quality are higher than organisations stated at figure 16.

No evidence found to support these criteria with particular lifecycle.

Figures 18 and 19 shows the results for organisation’s management style, question 10 (table 5) Figure 18 shows that, participants agreed on all statements on management styles

Figure 19 shows that, participants also agreed on all statements.

No evidence found to support these criteria with particular lifecycle.

Figures 20 and 21 shows the results for organisation’ emphasis, question 11 (table 6) Figure 20 shows that, participants are partially agreed on two statements (C and D).

Figure 21 shows that, participants agreed on all statements.

No evidence found to support these criteria with lifecycles.

Figures 22 and 23 shows the results for organisation’s definition of success, question 12 (table 7)

The collected data from participants’ responses of questions 7-12 has been analysed and the

result graphed as mentioned above. Table 8 below shows statistical analysis figures.

The table displayed the calculated figures to simplify the comparison between the two sectors.

The percentage figures show clear statistical evidence for some factors of the organisations

characteristics, leadership, management style/quality and emphasis for both sectors.

SA = Strongly Agree, A = Agree, N = Neutral, D = Disagree, SD = Strongly Disagree

Question Question type Public sector Private sector

7 Organisation's Characterisation SA = 20%

A = 32.5%

N = 22.5%

D = 25%

SD = 0%

SA = 15.8%

A = 39.5%

N = 22.4%

D = 14.5%

SD = 7.9%

8 Organisation's Leadership SA = 22.5%

A = 20%

N = 27.5%

D = 27.5%

SA = 2.6%

A = 35.5%

N = 35.5%

D = 17.2%


- 36 -

SD = 2.5% SD = 9.2%

9 Organisation's Management quality SA = 15%

A = 30%

N = 35%

D = 15%

SD = 5%

SA = 9.2%

A = 39.5%

N = 21%

D = 25%

SD = 5.3%

10 Organisation's Management style SA = 17.4%

A = 40%

N = 30%

D = 10%

SD = 2.5%

SA = 11.8%

A = 38.2%

N = 21%

D = 23.7%

SD = 5.3%

11 Organisation's Emphasis SA = 10%

A = 27.5%

N = 37.5%

D = 20%

SD = 5%

SA = 7.9%

A = 44.7%

N = 25%

D = 13.2%

SD = 9.2%

12 Organisation's Definition of success SA = 12.5%

A = 47.5%

N = 17.5%

D = 12.5%

SD = 10%

SA = 21.1%

A = 33.1%

N = 22.5%

D = 15.2%

SD = 8.1% Table 8: Summary result for questions 7 to 12

No evidence found that support the effects of these criteria on the use of development lifecycles.

Individual project investigation by SDLC

Section 2 in the questionnaire targets individual project investigation. This section will show the

analysis of the investigated projects. The investigation and the analysis at this section will use a

different approach from the one used in previous section. The collected data has been grouped by

lifecycle type (not sector type, as previous section). The analysis and results will be supported by

the published literature found in the references used for this report.


- 37 -

Table 9 show the participated organisations that used a combination of 2 lifecycles at one

project. (as given by participants)

Project Agile Incremental Iterative Waterfall No of

Iteration

1 Yes Yes 5

2 Yes Yes

3 Yes Yes

Table 9: Combination usage of SDLC

Investigated SDLC usage by individual project

27.6%

24.1%

31.0%

17.2%

34.5%

6.9%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

Agile Incremental Iterative Spiral Waterfall Other

Software Development Lifecycles (SDLC)

Perc

enta

ge

Figure 24: Investigated SDLC by individual project

Figure 24 show statistical figures for the investigated software development lifecycles (SDLC)

used in individual projects. The figure shows clearly that the majority of the participated projects

have used waterfall then iterative and so on.


- 38 -

Waterfall lifecycle

Waterfall is the first investigated method in the research. The tables and figures below show the results of the investigated projects that used waterfall

lifecycle.

Table 10 shows the result of the collected data from participants for the usage of waterfall and

techniques by project type, project size and organisation size.

Some of the shown results are confusing, for some organisations, participants responded to

project size with number of staff greater than the size of the organisation, such as project 1. Also

some participants did not provided any figures for project size or organisation size, as shown in

projects 4 and 9.


- 39 -

Project No Techniques Project type Project size Fulltime : Part time

Organisation size Permanent : Contractors

1 Prototyping, OO, Coding techniques

Data Mining 30 0 4 14

2 OO Intranet, Mainframe, Data mining

10 0 200 100

3 Prototyping, OO, Coding techniques, Structure analysis/design

Decision support 25 3 22 12

4 Prototyping, Coding, Structure analysis/design

Intranet, Management information system

Don’t know

120 20

5 OO, Coding techniques, Structure analysis/design, Cleanroom

Internet, Intranet, Management information system, Database

15 4 10000 1000

6 OO, Structure analysis/design

Transaction processing

10 0 6 0

7 OO, Coding techniques, Structure analysis/design

Intranet 25 10 450 80

8 Coding techniques, Structure analysis/design

Internet, Transaction processing, Decision support, Mainframe, Data mining

600 0 19000

Waterfall

9 Coding techniques, Structure

Internet Don’t know

Don’t know


- 40 -

analysis/design 10 Prototyping, OO,

Coding techniques, Structure analysis/design, Cleanroom

Internet, Transaction processing

4 1 285 80

Table 10: Waterfall & Techniques (by project and organisation sizes)


- 41 -

Concerning the determination of project completion, two questions (16 and 33 in the

questionnaire) have been set to collect data from participants, in order to find the best/suitable

determination methods that fit with waterfall, also these two questions will help in measuring the

quality in the intended product.

In both questions 16 and 33 I ask participants about the methods they used to determine the

completion of the project and the number of defects (also see table 14) in the project

respectively.

Figures 25 and 26 represent the results of the collected data from participants on questions 16

and 33 respectively.

Figure 25 shows that, one participant have used other method (criteria) than the methods used in

the questionnaire, as in project 8.

With correlation to table 13 below, the results shows that, project 6 and 7 were completed

projects, and the rest of the projects are still ongoing (underway) projects. So, project 6 used

“project in production” and project 7 used “customer signoff” methods, now with correlation

with figure 26, we have project 6 indicated the number of defects as“ between 5 and 20” and

project 7 as” defects between 20 and 50”. Therefore, none of the projects are free of defects, and

none of the projects have used more than one method to determine the project completion. Both

projects have neglected either customer satisfaction ”customer signoff” or project pass testing

and project is stable in production environment.

No evidence found to support these criteria.


- 42 -

Project 3

Project 4

Project 6

Project 10

Project 9

Project 1

Project 2

Project 4

Project 5

Project 7

Project 10

Project 8

Project in production Project in testing Customer sign off Other

Waterfall - Determine completion of projects

Project 10

Project 9

Project 8

Project 7

Project 6

Project 5

Project 4

Project 3

Project 2

Project 1

Benefits realised

Figure 25: Waterfall – determine the completion of projects

Project 1

Project 5

Project 9

Project 10

Project 6 Project 7 Project 8

Project 2

Project 3

Project 4

Less than 5 defects Between 5 and 20 bugs Between 20 and 50 bugs Between 50 and 100bugs

Other

Waterfall - Determine completion of projects by defects

Project 10

Project 9

Project 8

Project 7

Project 6

Project 5

Project 4

Project 3

Project 2

Project 1

Figure 26: Waterfall – determine the completion of projects by defects


- 43 -

In figure 26, 3 participants have responded with other, as follow:

1. Not applicable (N/A)

2. No metrics have been defined

3. Not sure of the exact number. But a large number.

Project complexity Concerning complexity in the projects, two questions (23 and 24 in the questionnaire) have been

set to collect (data) participants’ responses about complexity.

Table 11 below show the results of question 23. The complexity can be measured or determined

based on different factors, programming languages, technology used, the tools used, as well as

by asking participant directly about complexity using business rules and team experience/skills

etc.

In the questionnaire, I have used two ways (methods) to determine the complexity. First, the

programming languages used, beside the other factors in the table. Second, by asking a straight

forward about the complexity in the project using other project factors involved in the

development process.


The data shown in table 11 are exactly as given by participants.

Figure 27 below illustrates the results from question 24. It shows the level of complexity on each

project.

Projects Factors Tools No of tools

Management tools Microsoft Project / Excel 2

Automation testing tools

In-house development (Oracle PL/SQL based) 1

Case Modelling tools

Oracle Designer 1

Testing tools In-house development (Oracle PL/SQL based) 1

Programming language

SAS, Oracle PL/SQL, Java 3

Project 1

Design tools Oracle Designer 1


- 44 -

Open source tools

Source control tools

SVN 1

Other IBM Cognos 8.4 1

Total 11

Management tools MS Project 1



Visio 1

Testing tools


java, cobol,db2 stored procs 2

Design tools

Open source tools Spring 1


Clear case 1

Project 2

Other IBM websphere, DB2, harmony, web methods 4

Total 10



NUnit for component unit testing 1


Testing tools HP Quality Centre 1


C++, C# 2

Design tools

Open source tools Cruise Control build integration, FXCop code analysis

2


Subversion 1

Project 3

Other MS Visual Studio 1


- 45 -

Total 9

Management tools Used but not given 1



Testing tools


Java, .NET 2

Design tools

Open source tools


Project 4

Other

Total 3

Management tools Project 1


Rational Rose 1


Testing tools


C, C++, C#,PL-SQL 4

Design tools

Open source tools Git 1


Git

Project 5

Other

Total 7

Management tools Project 6 Automation testing


- 46 -

tools


Visio 1

Testing tools


Java, PL/SQL 2

Design tools

Open source tools


CVS 1

Other

Total 4



Mercury QC 1


Testing tools


Java, XML/XSL, JavaScript, etc 3

Design tools

Open source tools


CVS, Clear case 2

Project 7

Other

Total 7

Management tools



Testing tools

Project 8

Programming Cobol,MS.NET 2


- 47 -

language

Design tools

Open source tools


Other

Total 2

Management tools Used but not given 1


Used but not given 1


Testing tools Used but not given 1


Used but not given 1

Design tools

Open source tools


Project 9

Other

Total 4

Management tools Microsoft 1


Mercury 1


Rational 1

Testing tools Mercury 1


Java,Cobol,Objectsta,DB2 4

Design tools

Open source tools

Project 10



- 48 -

Other

Total 8 Table 11: Waterfall – Project complexity first criteria

The table shows a variety of complexity levels used within the projects. Also shows that, the

programming languages used in the projects are varied from new to old programming languages.

The selected programming languages used in projects indicates suitability of these languages

with the selected lifecycles, but no real evidence to support these criteria.

In figure 27 the colours of the bars reflects/indicates the selected answers (levels of complexity)

on each project, and the letters are to help reader to connect the graph to the statements in the

question, as shown in the question attached to the graph (question 24 in the questionnaire).

If we continue with projects number 6 and 7, we can see that, project 6 shows two levels of

complexity letter A and C, and project 7 have three levels of complexity, B, D and E, where E

indicates the lack of experience among the team members of the used lifecycle.

The fact, I couldn’t find any evidence that support the assumption criteria for waterfall lifecycle.

A

B

C

F

A

B

C

A

C

D

E

A

B

C

A

C

D

A

C

B

D

E

A

C

D

E

F A

C

E

1 2 3 4 5 6 7 8 9 10

Projects

Waterfall - Complexity of Projects

F

E

D

C

B

A

ABCDEF

Figure 27: Waterfall – Project complexity second criteria


- 49 -

Rate of change in the projects and constraints

1 2 3 4 5 6 7 8 9 10Projects

Waterfall - Project requirements defined at the beginning of the project (including user andbusiness requirements)

Complete

Ambiguous

Partial

Incomplete

Figure 28: Waterfall – Projects requirements defined at the beginning of the project

Figure 28 shows the results of question 25. The coloured bars reflect the responses collected

from participants on the investigated projects. It shows the rate of change in the project by

defining the requirements status at the beginning of the projects. The criteria I have used in the

question are, received complete requirements or partial or incomplete or ambiguous.

The graph illustrates the project number with its requirements status. For example, if we look

again to the completed projects number 6 and 7, we see that project 6 has received partial

requirement at the beginning of the project, and project 7 has received incomplete requirements

at the beginning of the project.

No evidence found to support these criteria on waterfall lifecycle.

Proceeding with the analysis, table 12 (appendix J) shows the questions that used in the

questionnaire to rate the changes in the projects. Questions 26 to 29 will provide clear picture of

the data illustrated at figure 29. The figure shows 4 rows and 10 columns (column 9 is empty has


- 50 -

no data), every row has a unique colour which reflects one question of the 4 questions mentioned

above. Again if we look to projects number 6 and 7, we conclude the following:

Project 6 has the following responses on the questions: Q26 = 0, Q27 = 1, Q28 = 1 and Q29 = 0

Project 7 has the following responses: Q26 = 12, Q27 = 12, Q28 = 0 and Q29 = 10.

The number with a plus (such as 100+), indicates that the number of requests received by any of

the 4 questions is in the hundreds, which mean, participants did not response with exact number,

instead they respond with ‘hundreds’ of requests. The same is for (1000+), participants indicates

that the requests for changes received are in thousands, such as for project 8.

The data collected about project 8, shows high rate of changing in the project, the project is

struggling and have difficulties in controlling requirements and controlling development process.

We will look closely to this project. First, participant did not response to question 50 in the

questionnaire, which could give more details about the methodology in use. Second, the

programming languages used in the project are COBOL and MS.NET (see table 11), which is

mixed of old and new programming languages (this is reflect complexity in the project). Third,

the expected level of quality indicated by participant is very high quality (see figure 30). Fourth,

the duration of the project is 6 years (see table 13), and number of staff working on the project

are 600, which indicates the project is huge.

Clearly this project needs high management/leadership experience; high team experience,

especially in development methodologies, it may need change to the development methodology.

The fact, this project needs special prescription.

Does waterfall lifecycle capable to handle such huge changes in the project?

Is waterfall suitable for such project?

The fact, I couldn’t find any evidence that could answer or to support these criteria.


- 51 -

13100026100+

15

080100102

2

1000+

20154

3100+100+

20+

3

1000+

1201874100+100+100+

1 2 3 4 5 6 7 8 9 10

Projects

Rate of change in project

Question 26

Question 27

Question 28

Question 29

Figure 29: Waterfall – Rate of change in projects by request

Waterfall - Levelof quaity expected in the project

1

1

11

1

1

1

1

1

1

1

2

3

4

5

6

7

8

9

10

Very high quality

High quality

Good quality

Low quality

No quality

Figure 30: Waterfall – Level of quality expected in the projects


- 52 -

Figure 30 show project quality result for the investigated projects, the colour and the number at

the top of the axis refer to the project number under investigation. At the centre of the chart the

quality starts at zero and then it increase till reach to the heads of the axes, where the indicators

reach to very high quality with a value of one next to the indicator.

So as I’ve mentioned above, the quality of the produced product (system/application) can be

measured and determined by different factors, the number of defects in the final product, by

project completion methods (e.g. project signoff, project in production) and by asking

participants (question 32) directly about quality using other factors. These criteria should give

slight indication if waterfall will fulfil customer’s needs/requirements (mostly when the project

has somehow a stable requirements), and if customer will be satisfied with the final product.

It is clearly that, the demand on very high project quality does not prevent or reject the use of

waterfall lifecycle. It is very common and in many cases, that software development projects do

not deliver the quality of the product as required by the customer. “Customer, often complain

that their real requirements are hardly met and that the software is too difficult to use” (27).

Table 13 shows the status of the investigated waterfall projects and its durations, by project

number.

Project No Project status (completed ‘YES’ or not completed ‘NO’)

Project duration (months)

1 NO 3 years

2 NO 6 months

3 NO 3 years

4 NO 8 years

5 NO 1 year

6 YES 1 year

7 YES 18 months

8 NO 6 years

9 NO Not given

10 NO 6 months Table 14: Waterfall – Investigated project status and duration


- 53 -

Table 14 will support figure 25 with more details, also it gives an indication about the quality of

the project with correlation (support) to figure 30 above.

Less than 5 defects

Between 5 and 20 bugs

Between 20 and 50 bugs

Between 50 and 100 bugs Other

1 Yes 2 N/A 3

No metrics have been defined

4

Not sure of the expect number, but a large number

5 Yes 6 Yes 7 Yes 8 Yes 9 Yes 10 Yes

Table 14: Waterfall – Expected number of defects at the completion of project

Project criticality

Table 15 shows how critical is the failure of the project to the organisation. We can see that

projects 3 and 5 are very critical to the organisation and the consequences of the projects failure

are potential loss of human life.

Found that, waterfall lifecycle is suitable for high critical projects.

Damage to reputation

Serious loss of income

Potential loss of life

Nothing much Other

1 Yes

Budget/Planning/Senate Estimates incorrect - decisions made on incorrect information

2 Yes 3 Yes 4 Yes 5 Yes 6 Yes 7 Yes 8 Yes Yes 9 Yes 10 Yes Yes

Table 15: Waterfall – How critical is the failure of project to the organisation


- 54 -

Waterfall - Projects budget

Project 1

Project 2 Project 3

Project 4

Project 5

Project 6

Project 7


0 1 2 3 4 5 6

Generous budget

Sufficient budget

Insufficient budget

No budget

Don't know

Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Project 9

Project 10

Figure 31: Waterfall – Projects budget

Figure 31 illustrating the result for project’s budget for the investigated projects. The important

of these data are indirectly reflects the important of the project to the organisation as well as if

the budget adequate to achieve business requirements and will help to produced project

outcomes. “Project sponsors are often frustrated because the software is delivered late and/or

over-budget” [27]. The graph shows the budget status of the projects under investigation as well

as the projects names at each level (e.g. project 1, project 2 etc).

In any project, project’s budget has an influence on all project factors including the development

lifecycle.

Let’s have a look at some of the data presented at the graph, participant of project 4 did not know

the budget of the project. Projects 1, 5 and 7, the three projects have insufficient budget, and with

correlation with figure 30, the quality of the three projects are expected to be high, very high and

good quality respectively.

I couldn’t found evidence that support the influence of the budgets on the lifecycle. But, it is

obvious that the projects with insufficient budget will have a lot of problems, and the affects


- 55 -

could occur on different factors of the projects such as, the number of staff working on the

project, changing the delivery date, work load, technology, quality etc.

In additional to the above, the success of the project phases on time (gathering requirements,

design/architectures, implementation etc, will take the project to the safe end, “preserving the

project from budget overruns” [27].

Team experience and communication Figure 32 shows the level of experience of the project’s team. The legend consists of project s names differentiated by different colours. The more experienced is the team, the more successful of using the development methodology.

The result shows clearly, none of the teams has little or poor experience.


Waterfall - Team experience


Project 4

Project 5

Project 6

Project 7

Project 8

Project 9 Project 10

0 1 2 3 4 5 6

Very high experience

High experience

Good experience

Little experience

Virtually no experience

Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Project 9

Project10

Figure 32: Waterfall – Project team’s experience


- 56 -

Waterfall - Team empowerment on the projects

Project 1

Project 2

Project 3

Project 4

Project 5 Project 6

Project 7

Project 8

Project 9

Project 10

0 1 2 3 4 5 6

Always

Often

Rarely

Sometimes

Never

Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Project 9

Project 10

Figure 33: Waterfall – Team empowerment

Following the teams experience results, another question has been set to collect data from

participants to measure the level of team’s involvement in decision making within the project.

Figure 33 shows the result of the collected data from question 36 in the questionnaire. The

graph shows the team’s involvement level in decision making, the legend shows projects names.

From the analysed results shown, project 1, 2, 3, 4, 5, 6 and 7 have responses with “not always”,

which mean their teams are not always involved in making decisions when there is a problem in

the project.

If we assume that team empowerment should be measured base on individual’s role in the

project, the result show that this is not the case, because the leadership of projects 1, 5 and 7 are

team leaders, and looking to projects 8 and 10, the result show “always”, with correlation to

figure 37, we see that, their roles are “architect” for project 8, and “project manager", team

leader, programmer etc” for project 10. Therefore, this assumption is not correct.

Another assumption is that, the leadership policy of the organisations has an influence on the

structure of the teams in away that it could give a member more empowerment than other. If this

is the case I must stated that, the organisation’s management style should be of both, leadership


- 57 -

and collaboration. Project managers and team leaders should be acting in a facilitating (e.g.

flexibility etc) role with team involvement (empowered) in decision making.


0 1 2 3 4 5 6

Number of meetings fortnightly

1

2

3

4

5

6

7

8

9

10

Waterfall - Team meetings (fortnightly)

Figure 34: Waterfall – Frequent team meeting (fortnightly)

Figure 34 shows the frequent team meeting (fortnightly). For example, project 8 meets 4 times a

fortnightly. The frequent team meeting is very important to the project. Team meetings help and

support in:

1. Highlight issues before turn into problems

2. Improve team’s communication

3. Share knowledge among team members

4. Address programming, technological and risks issues etc

5. Agree or not to agree on particular issues

6. Improve productivity


- 58 -

Waterfall - Common communication methods among team members

Project 1 Project 2

Project 3 Project 4

Project 5

Project 6

Project 7 Project 8

Project 9

Project 10

0 1 2 3 4 5

Formal communication

Mostly formal

Mixture of formal and informal

Mostly Informal

Informal communication

Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Project 9

Project 10

Figure 35: Waterfall – Common communication methods among team members

Figure 35 shows the communication methods used between team members. (refer to question 38

in the questionnaire).

Team communication and frequent meetings help the software team in controlling process and to

achieve effective delivery of product.

One of the main reasons for using a software development methodology is to support and

enhance communication among stakeholders during all of the development phases [32].


- 59 -

Waterfall - Common communication methods between team members and managers

Project 1

Project 2


Project 6

Project 7 Project 8

Project 9

Project 10

0 1 2 3 4 5


Mostly formal


Mostly Informal


Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Project 9

Project 10

Figure 36: Waterfall – Common communication methods between team members and managers

Figure 36 shows the communication methods between team members and managers. (refer to 39

in the questionnaire).

For both figures 35 and 36, the result shows that, the methods used in all projects are mixture of

formal and informal communication, which indicates that waterfall is suitable for both methods,

even if some of the projects are encountering some issues. Therefore, for some projects the

communication methods needs to be improved, if we take the fact that, face-to-face

communication is more preferred than indirect communication because this is supposed to be

more beneficial.

In additional to the above, I did not found more evidence that support the communication

methods for waterfall lifecycle.


- 60 -

Two questions (40 and 41) are designed to collect data from participants about leadership’s

role(s) and the corresponding experience, such as project managers, team leaders, programmers,

analysts, designers and architects etc, figure 37 shows the results of the collected data from the

two questions stated above.

First, it is clearly that project 9 shows that participant’s role on project is tester (testing roles),

therefore, this project will be excluded from the analysis, because the research objective is

targeting the development methodologies in software development, and the questionnaire

seeking responses from individual with experience in development methodologies. Second,

project 1 has indicated 5 roles and at the same time, has no previous experience (first time act on

these roles). Third, two participants have maximum experience of 10 years, and two participants

with only 6 months of experience. The rest vary from 2 years to 7 years of experience.

Therefore, it seems that waterfall works with all types (little, medium and high) of experience.

Waterfall - Project management/Leadership main roles on projects

0 1 2 3 4 5 6 7

1

2

3

4

5

6

7

8

9

10

Expe

rinec

e &

Pro

ject

No

Number of roles

Programmer

Designer Analyst

Tester

Team Leader

Business Analyst

Project Manager

Architect

Other

3 years

10 years

7 years

6 years

6 months

10 years

6 months

First time

2 years

First time

Figure 37: Waterfall – Project Management/Leadership main roles and experience


- 61 -

Figure 38 show leadership experience in software development field, business domain and

development lifecycles. These roles are important and have an affect on project progress. Three

Questions (42, 43 and 44) in the questionnaire are designed to collect data from participants.

The results shown in the graph are straight forward. First assumption is confirmed, as mentioned

above, project 9 has no experience in development lifecycle and will be excluded from this

analysis. The vast majority of the participants have average experience in waterfall development

lifecycle of 5 years, if we take into count that the minimum experience is 3 years and the

maximum is 13 years. Nevertheless, the importance of leadership experience to the development

team and the organisation can’t be underestimated. The result also shows the vast majority of

participants have experience above the average (5 years) in software development field.

These improve and increase the validity of the research, that 7 of the 9 responses (if we excludes

response 9) were experienced participants.


Waterfall - Project Management/Leadership experience

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

1

2

3

4

5

6

7

8

9

10

Proj

ect N

o

Number of years

Software development fieldexperience

Business domain experience

SDLC experience

Figure 38: Waterfall – Project Management/Leadership experience


- 62 -

Figure 39 shows the techniques used in projects. The techniques used in the questionnaire are the

common techniques used in industry. The result shows that, most of the participated projects

agreed on the techniques that suitable with waterfall lifecycle.

As this research is focusing on the development methods (lifecycles) and techniques used with

them, studies of individual techniques in more details is another area of study, this is should be

addressed in future study.

Prototyping OO Coding Structure Analysis & Design

OO

Prototyping OO Coding Structure Analysis & Design

Prototyping Coding Structure Analysis & Design

OO Coding Structure Analysis & Design Cleanroom

OO Structure Analysis & Design

OO Coding Structure Analysis & Design

Coding Structure Analysis & Design

Coding Structure Analysis & Design

Prototyping OO Coding Structure Analysis & Design Cleanroom

1

2

3

4

5

6

7

8

9

10

Proj

ects

Waterfall - Techniques used in projects

Prototyping

OO

Coding

Structure Analysis &Design

Cleanroom

Pair Programming

Techniques

Figure 39: Waterfall – Techniques used in projects

Concerning the advantages and disadvantages of the lifecycle used in projects, two questions has

been set to collect data from participants, question 48 and question 49 in the questionnaire. The

results for both questions are shown in figure 39 and figure 41 respectively.

Figure 40 below illustrates the advantages of the waterfall lifecycle used in the investigated

projects. The graph is very straightforward.

Description of both figures, the letters A, B, C, D etc, represent the statements in the questions

that the graphs reflects. The letter K reflects participant responded with ‘other’


- 63 -

Waterfall - Advantages/Benefits of the used lifecycles

A

A

A

A

A

A C

D

E

F

F

F

F

F

F

G

G

G

G

H

H

H

I

J

J

J

J

J

K

0 1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

9

10

Proj

ect N

o

Advantages

A

B

C

D

E

F

G

H

I

J

K

Figure 40: Waterfall – Advantages/benefits of the used lifecycles

For project 2, participant responded with ‘None’ to this question, letter K reflect the answer (K = None).

A B C D E F G H I J

K


- 64 -

A C G

A B D K

A B C E

A B C E H J

B G H

G H J

A G I

A

A D G H I

F K

0 1 2 3 4 5 6

Disadvantages

1

2

3

4

5

6

7

8

9

10

Proj

ect N

oWaterfall - Disadvantages/Limitations of the used lifecycles

A

B

C

D

E

F

G

H

I

J

K

Figure 41: Waterfall – Disadvantages/limitations of the used lifecycles

Project 1 responded with K = low levels of productivity, lack of visibility in the project

impediments, lack of experience in the existing systems, hard to maintain and integrate existing

systems, difficult to track and measure progress

Project 10 responded with K = Potential issues may be identified late in the lifecycle

Figure 41 illustrates the disadvantages of the waterfall lifecycle used in the investigated projects.

The graph is very straightforward. The letter K in the graph reflect the participant’s answer

‘other’


- 65 -

Last question in the questionnaire is question 50. I have asked participants “under which

condition particular development methodologies should not be used”. The responses collected

from this question are illustrated in table 16 (appendix H) as answered by participants:

Note: two participants (8 and 9) did not provided answer to question 50.

Findings – Waterfall lifecycle

My founding will be based on the software development lifecycle under investigation. As we

have seen in the above section, the analysis carried out was for waterfall lifecycle, which is the

first lifecycle investigated. Following my explanation and analysis I will list what I have found

as follow:

1. I’ve found that the vast majority of the participated organisations are using one or more

of the development methodologies (lifecycles and techniques) used in the questionnaire.

2. Waterfall has been used at different levels. It has been used with big projects as well as

with small projects. Also has been used within different organisation sizes (see table 10).

A B C D E F G H I J

K


- 66 -

3. Waterfall has been used for different types of projects, such as Internet, Intranet,

Management information system, Transaction processing, Decision support, Database,

Mainframe and Data mining (see table 10).

4. Waterfall has been used with different length of project, projects length varied from 6

months to 8 years (see table 13).

5. Waterfall lifecycle is suitable for high critical projects (table 15).

6. Found that, six participants have less than 5 years of experience in the main roles of the

project (see figure 36). Lack of experience should be investigated in future study. As for

some, they assume that people without experience go with waterfall.

7. Found that, the vast majority of the projects requirements received at the beginning of

projects are received either in partial or incomplete or with ambiguity (see figure 27). It is

well known that one of waterfall weakness is lack in controlling changes in requirements.

8. In additional to the found above, in many cases (projects), the rate of changes in projects

requirements are high, which it cause delay, confusion, not progressing, and could lead to

budget overrun (see figure 28).

9. Seven out of ten participants have indicated in their responses that teams are not always

involved in decision making.

10. Projects with high quality, are more likely to use waterfall, but hard to produce it.

11. The well established requirements, leads to good quality product

12. Waterfall works well with stable requirements

13. I have found no evidence to support the ideas that the project factors such as size, time

pressure, complexity, quality, budget and technology affects the use of waterfall method.

14. I have found no evidence to support many of the founding mentioned above.

15. The results founded from advantages and disadvantages questions for the used of

waterfall lifecycle as follow (see figures 40 & 41), where participants agreed on them:

Advantages a. Six responses indicated that waterfall has more control of the development process.

b. Six responses indicated that waterfall deliver good quality product


- 67 -

c. Five responses indicated that waterfall produce product with less bugs/defects

d. Four responses indicated that waterfall improve communication between team members

e. Three responses indicated that waterfall improve communication with customers/clients

f. But, none of the responds agreed on that, waterfall increase productivity Disadvantages

a. Seven responses indicated that waterfall has difficulties to control changes in requirements

b. Five responses indicated that waterfall has difficulties in controlling iterations c. Four responses indicated that waterfall has difficult to meet client/customer

requirements/needs d. Four responses indicated that waterfall cause delays to delivery of the product e. Three responses indicated that waterfall can’t prioritise requirements

When to use and not to use waterfall, based on the analysis

Use Waterfall Not to use Waterfall

For repeatable projects, very similar in

requirements, design, technology etc.

“For example, if a company has experience

in building accounting systems, I/O

controllers, or compilers, then building

another such product based on the existing

designs is best managed with the waterfall”

[33].

If project is not going well

To gather requirements (requirements

elicitation), waterfall consumes too much

time

If project has complete requirements or rate

of changes are small.

If project requirements are not complete

and rate of changes in requirements is high,

or the project scope or project requirements

are subject to big changes.

When requirements gathering / signoff is


- 68 -

ineffective.

When customers doesn’t understand their

needs or the expected outcomes

With iterative approach As standalone

Finally, after this investigation on waterfall lifecycle and on the light of the analysis stated

above, for the projects intended to use waterfall, its use should be limited to situations

where the project requirements and the implementation of those requirements are very

well understood by both project team and customer/user.


- 69 -

Agile lifecycles

Agile is the second investigated method in the report. Projects description The tables and figures in this section illustrate the results of the investigated projects that used

agile methods.

Table 17 below, shows the number of participants that used agile methods and the techniques

used with each method, project type, project size and organisation size. The results show that, the

sizes of the projects are between small and medium sizes projects. One participant did not

provide with the organisation size, in which I assume (from the size of the project) its big

organisation.

The result also shows that, there are other agile method used, other than the ones used in the

questionnaire such as Rational Unified Process (RUP) method. Also there are some other

techniques used other than the ones used in the questionnaire such as Story Points, Backlog,

Story-driven development and Proprietary methodology techniques.

The investigation was not limited to a certain agile methods, it is included all the agile methods

currently used in industry.

The fact, as agile methods are a set of software development methods, I believe agile methods

should be investigated separately (standalone research).

Clearly the result shows that there are some sorts of agreement among participants on the

techniques used with agile methods. The techniques used in the questionnaire are common

among the vast majority of the projects.


- 70 -

Project

No Methods Techniques Project type Project size

Fulltime : Part time


1 Architecture centric approach (such as Rational Unified Process)

Prototyping, OO, Pair programming, Story Points, Backlog

Intranet, Document management

30 6

2 SCRUM Prototyping, OO, Coding techniques, Structure analysis/design, Pair programming

Internet 6 0 4 0

3 Lean Software Development (LSD)

Prototyping, OO Intranet, Training system

6 1 5 2

4 Proprietary methodology

Prototyping, OO, Coding techniques

Intranet 4 2 30 10

5 Extreme Programming (XP)

Prototyping, Coding techniques

Communication 7 2 0 8

6 SCRUM Prototyping, OO Internet, Database 10 8 7 13

7 Extreme Programming (XP)

Prototyping Internet, Intranet, Management information system, Database

15 4 10000 1000

Agile

Projects

8 Methods used at individual projects: Extreme Programming (XP),

Prototyping, OO, Coding, Structure analysis/design, Pair programming

Management information system,

Decision support,

Sensor data fusion

22 1 24 0


- 71 -

SCRUM,

Personal software process,

Team software process.

9 SCRUM Prototyping, OO, Coding techniques, , Structure analysis/design, pair programming

Internet, Transaction processing, Database

10 5 4 2

10 SCRUM Prototyping, OO, Pair programming, Story-driven development

Internet, Intranet, Management information system, Data mining

35 5 200 300

11 SCRUM Prototyping Internet 100 5 50 50

12 Watered-down SCRUM, BDD

Prototyping, OO, Coding techniques,

Internet, Database 1 0 15 0

13 SCRUM, XP Internet, Transaction processing

4 1 285 80

Table 17: Agile & Techniques (by project and organisation sizes)


- 72 -

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Project 1

Project 3

Project 5

Project 6


Agile - Determine completion of projects

Project 8

Project 7

Project 6

Project 5

Project 4

Project 3

Project 2

Project 1

Figure 42: Agile – determine the completion of project

Project 2

Project 5

Project 7

Project 1

Project 3

Project 4

Project 8

Project 6

Less than 5 defects Between 5 and 20bugs

Between 20 and 50bugs


Other

Agile - Determine completion of projects by defects

Project 8

Project 7

Project 6

Project 5

Project 4

Project 3

Project 2

Project 1

Figure 43: Agile – determine the completion of projects by defects


- 73 -

The results of the 2 figures stated above, concerning the quality of the investigated projects. Refer

to table 19 below, the results shows that, only project 5 and project 8 were completed and the rest

are underway projects. Project 5 was 6 months duration and project 8 was 11 months duration.

Concerning the quality of the produced product (system/application), 3 questions (refer to

questions 16, 32 and 33) in the questionnaire have been set to collect data from participants, in

order to measured and determined product’s quality. The quality of the produced product

(system/application) can be measured and determined by different factors, the number of defects in

the final product, by project completion methods (e.g. project signoff, project in production) and

by asking participants directly about quality using other factors. These factors should give slight

indication if waterfall could fulfil customer’s needs/requirements (mostly when the project has

somehow a stable requirements), and if customer will be satisfied with the final product.

Figure 42 show that, the majority of the projects have chosen 2 methods to determine the

completion of the projects, where the two methods reflect or intend to achieve the highest possible

quality of the final product by meeting customer’s needs/requirements and technically the product

is in production environment.

Figure 43 show that, the quality of the expected product is high for all projects.

If we look to the completed projects, projects 5 and 8, it can be seen that project 5 has used two

methods, project in production and customer signoff, with a number of defects less than 5, and

project 8 has used, project in production method with a number of defects between 20 to 50

defects.

Now if look to the result of the direct question (question 32) which illustrated in figure 47, result

show that project 5 completed with high quality product, and project 8 completed with good

quality.

Therefore, from the results we have slight evidence, that:

• Product in production and customer signoff are successful methods to use to determine the

completion of the projects, in which they satisfy customer (teams were satisfied with these

methods).


- 74 -

• Agile methods dose not (or hard) to produce product free of defects, which contrary to the

impression we get when we read publications of advocates of the agile movement, such as

[34].

• Agile methods in some cases can fulfil customer’s requirements

The project objective is not to compare methods (Agile with waterfall, Spiral etc). This is good to

be included in future study.

Project complexity Concerning complexity in the investigated projects, two questions (question 23 and 24 in the

questionnaire) have been set, asking participants to provide data about complexity in projects.

Table 18 below shows the result of question 23.

The complexity can be measured or determined based on different factors. I have used two ways.

First, asking participants about the tools used in the projects such as, programming languages used,

the technology or the combination of technologies used and the tools of testing, management tools,

design etc. Second, by asking participants directly about the rate of complexity in the project using

other project factors involved in the project development.

The data shown in table are exactly as given by participants. Figure 44 below illustrates the results

from question 24. It shows the level of complexity of each project.

Management tools MS Project


Test Complete

Case Modelling tools Enterprise Architect, Visio

Testing tools HP Quality Centre

Programming language .NET

Design tools Enterprise Architect

Open source tools

Source control tools Team Foundation Server

Project 1

Other XML Authoring, XMLSpy


- 75 -

Management tools Agilo


Testing

Case Modelling tools Webdriver

Testing tools

Programming language Java

Design tools UMLet

Open source tools Eclipse, SoapUI, Jboss, Seam, Hibernate

Source control tools Subversion

Project 2

Other

Management tools MS Project, PMBoK, Jira, Confluence


Load Runner

Case Modelling tools UML / Enterprise Architect

Testing tools Mercury

Programming language Java, XML

Design tools

Open source tools Several


Project 3

Other




Testing tools SilkPerformer


Design tools

Open source tools Jira

Source control tools CVS

Project 4

Other


- 76 -

Management tools Microsoft


TFS build


Testing tools Hudson

Programming language C#,T-SQL


Open source tools

Source control tools TFS

Project 5

Other ORM

Management tools Prince 2



Testing tools


Design tools Rational Architect

Open source tools


Project 6

Other

Management tools Basecamp, Omni Project



Testing tools Spec

Programming language Ruby

Design tools

Open source tools Ruby Gems, Sphix

Project 7

Source control tools Mercurial


- 77 -

Other Table 18: Agile - Project complexity first criteria

The complexity can not be directly determined, it is based on factors such as novelty of the

technology, the novelty of the business problem, team skills and the combination of technologies

used [35]. The table above shows a variety of complexity used within the projects. Also it shows

the programming language used in the projects, Java, .NET, XML, C#,T-SQL, Ruby. This shows

that the programming languages used are the common programming languages, and the vast

majority of them are new languages. The languages do not lead to high complexities in projects.

In figure 44 the colours of the bars reflects the selected answers (levels of complexity) by projects,

question 24 is attached to the graph (refer to question 24 in the questionnaire).

Graph description: the level of (A, B, C, etc) complexity shown in the bars, reflect the statement

selected in answering the question, which is corresponding the marked statements in the question.

I couldn’t find any evidence that support these assumption criteria.

Due to the limitation of the project objectives, and the limitation in the project time, further study

need to be done on the complexity factors of agile projects.

A

B

C

D

E

A

B

C

D

E

A

C

A

B

C

D

E

A

B

C

D

E

A

B

C

E

F

A B

C

D

1 2 3 4 5 6 7 8

Projects

Agile - Complexity of Projects

F

E

D

C

B

A

ABCDEF

Figure 44: Agile – Project complexity second criteria


- 78 -

Rate of change in the projects and constraints Figure 45 shows the results of question 25. The coloured bars reflect the responses collected from

participants on the question for investigated projects. It shows the rate of changes in the project by

defining the requirements at the beginning of the projects, such as requirements are received

completely or in partial etc. The results show that none of the projects have received complete

requirements at the beginning of the project.

I could not find evidence to support these criteria.

Proceeding with the analysis, table 12 (appendix H) shows the questions that addressing the rate of

changes in the projects. Figure 46 show the results of the collected data from questions 26 to 29.

The figure shows four rows and eight columns (3 rows of column 3 are empty, no data given), the

column number also refer to participated projects, every row has a unique colour which reflects

one question of the four questions mentioned above.

The numbers on the top of the bars are for requested of changes received on projects.

The number with a plus (such as 100+), indicates that the number of requests received is in

hundreds, which mean, participants did not respond with exact number, instead they responded

with ‘hundreds’ of requests. The same is for (12+), participants indicates that the requests for

changes received are in dozens, such as project 2.



- 79 -

1 2 3 4 5 6 7 8Projects

Agile - Project requirements defined at the beginning of the project (including user andbusiness requirements)

Complete

Ambiguous

Partial

Incomplete

Figure 45: Agile – Projects requirements defined at the beginning of the project

12

34

56

78

50

1023 25 6

100

30

12+

100+

6

200

20

12+

100+

12

34

56

78

Projects


Question 26

Question 27

Question 28

Question 29

Budget is re-established every year

Increasing scope of material covered by the system

Additional functions added to a component

New staff.Request to start a second project prior to completion of this project.

Figure 46: Agile – Rate of change in projects by request


- 80 -

Agile - Levelof quaity expected in the project

0

00

0 0 00

0

00 0

0000

0

0 00

0000

0 0 00

0000

01

1

1

1

1

1

11

1

2

3

4

5

6

7

8

Very high quality

High quality

Good quality

Low quality

No quality

Figure 47: Agile – Level of quality expected in the projects

Figure 47 show projects quality. The colours and the numbers at the top of the axes refer/reflects

the investigated projects. At the centre of the chart the quality starts at zero and then it increase

until it reach to the heads/top of the axes, when the indicators reach to the value of very high

quality, corresponding to the legend.

“The agile methods are considered to be a reaction against perspective traditional methodologies

and a response to the need for quality, rapidly developed systems“ [36].

It is clearly that, the demand on high quality projects does not prevent or reject the use of agile

methods in software development. It is very common and in many cases, that software

development projects do not deliver quality product as intended or required by customer.

“Customer, often complain that their real requirements are hardly met and that the software is too

difficult to use” [27].

Table 20 shows the status of the investigated agile projects and its durations, by project number.


- 81 -

Project No Project status (completed = ‘YES’ or not completed = ‘NO’)


1 NO 60 = 5 years

2 NO 24 = 2 years

3 NO 6 months

4 NO 12 = 1 year

5 YES 6 months

6 NO 96 = 8 years

7 NO 36 = 3 years

8 YES 11 months Table 20: Agile - Investigated project status and duration

Project criticality

Table 21 shows how critical is the failure of the project to the organisation. We can see that

projects 7 and 1 are very critical to their organisations and the consequences of their failure could

lead to potential loss of human life and damage to the government reputation respectively.

In terms of prove the assumption that, “agile methods are not considered suitable for highly critical

projects” [19]. It is clearly that the only project with no criticality is project 8. Other projects are

critical either to the development team or the organisation.

This is contrary to the assumption stated above, which mean agile methods are suitable for high

critical projects.

No other evidence found to support these criteria




Nothing much Other

1

Possible public damage to government reputation

2 Yes Yes 3 Yes 4 Yes Yes Loss of capability 5 Yes Yes Ministerial loss of face 6 Yes Potential legal issues 7 Yes Yes 8 Yes

Table 21: Agile – How critical is the failure of project to the organisation


- 82 -

Agile - Projects budget


Project 4

Project 5

Project 6

Project 7

Project 8

0 1 2 3 4 5 6

Generous budget

Sufficient budget

Insufficient budget

No budget

Don't know

Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Figure 48: Agile – Projects budget

Figure 48 represent the result of project’s budget. The important of these data are indirectly

reflects the important of the project to the organisation as well as if the budget adequate to achieve

project requirements and will help to produced project outcomes. “Project sponsors are often

frustrated because the software is delivered late and/or over-budget” [27]. The graph shows budget

status of the projects as well as the project name at each level (such as project 1, project 4 etc).

Project budget has an influence on the development methods and on the project outcomes.

“Preserving the project from budget overruns” [27].

Let’s have a look at projects 4, 6 and 8, the three projects have insufficient budget. The three

projects indicated the expected quality of final product as high quality, high quality and good

quality respectively.

I couldn’t found evidence that support the influence of project budget on agile methods. But, it is

obvious that projects with insufficient budget will have a lot of influence and maybe “problems”

on projects (factors) progress. Many factors in the projects will be affected, such as, the number of

staff working on projects, changing delivery date, technology and quality etc.


- 83 -

The success of project phases on time (setting requirements, design/architectures, implementation

etc) will take the project team to satisfied end.

Team experience and communication Figure 49 shows the level of experience of the project’s team. It shows the level of experience for

each project’s team as well as the number of projects at each level.

The more experienced team, the more successful of using agile development methods.

“Knowledge and background of the traditional techniques and methodologies are considered

beneficial before using agile method (XP), because it gives student a great appreciation of the

purpose of XP and its process” [37].

The result shows clearly that, none of the teams has little experience. It can be seen that the

average of teams experience is high. “Agile methods are suitable for small teams of 2-10

developers” [38], and suitable for “small projects, or projects that can be divided into sub-projects

suitable for small teams” [35].

Agile - Team experience

Project 1

Project 2 Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

0 1 2 3 4 5


High experience

Good experience

Little experience


Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Figure 49: Agile – Project team’s experience


- 84 -


participants to measure the level of team’s involvement in decision making on projects.

Figure 50 below shows the result of the collected data from question 36 in the questionnaire. The

graph illustrates the number of projects against the team involvement level. The colours in the

legend are to connect projects to the corresponding level.

From the analysed results shown, it look like that projects 1, 2, 4, 5 and 8 have responses with

‘always’, which mean their teams are always involved in decision making when there are problems

in projects.

If we assume that the result show in the graph, about the team empowerment as always good,

because the teams are small, and communication methods playing good roles in connecting the

project team members together (be active project member, understanding and paying respects to

each other) and be an important member within the team.


Agile - Team empowerment on the projects

Project 1 Project 2

Project 3

Project 4 Project 5

Project 6

Project 7

Project 8

0 1 2 3 4 5 6

Always

Often

Rarely

Sometimes

Never

Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Figure 50: Agile – Team empowerment


- 85 -

Figure 51 shows the result of team meetings (fortnightly). For example, project 5 meets four times

fortnightly. The frequent team meeting is very important to project progress.

I believe that, the needs for frequently team meeting, such as twice a week is necessarily,

especially when the project is too complex, requirements are not well defined, project not

progressing well, when project team members in needs to share knowledge and experience and

when a risk issues occurs.

Team meetings help and support in:

1. Highlight any issues before they turn into problems

2. Improve communication among the team members

3. Share knowledge among team members

4. Address programming, technological, technical and risk issues etc

5. Agree or not to agree on some issues

6. Improve productivity

No evidence found to support these criteria

0 1 2 3 4


1

2

3

4

5

6

7

8

Agile - Team meetings (fortnightly)

Figure 51: Agile – Frequent team meeting (fortnightly)


- 86 -

Figure 52 shows the results about communication methods used between team members. (refer to

question 38 in the questionnaire). The graph shows the communication methods used by every

project. Team communication and meetings help the development team to achieve effective

delivery of product.

One of agile method (XP) principles is “open, honest communication” [39] among team members

and with customer.

The fact, concerning communication and frequent meetings in order to achieve project’s goal, I

think user involvement must be considered, as user has a positive affect on the development team.

User/customer must be given the opportunity to be more active participant and even part of the

project team, by introducing new communication techniques such as development work shops,

prototyping sessions and development walkthrough. “User-driven development became a

possibility during this time” [28].

User/customer involvement in project development become the area of interest and must be

included in future researches/study. As customer participation in development is so important (not

in all cases), it is well assumed that customer involvement in development will improve

communication between team members and customer, and eventually delivering a quality product

that satisfy customer and the development team.

However, still there is no agreement among experts of having customer on the development site,

and some still arguing on this. “User participation is not better in all cases, and user satisfaction

can depend on many other factors“ [40].

“User involvement in the development team is an important factor in which stakeholders can

participate in development” [29].

There are many reasons given behind customer/user involvement in development:

1. Improve the quality of the developed application/system, by improving the understanding

of the solution and avoiding the development of the unacceptable solution [29].

2. To avoid unrealistic customer expectation [30].

3. To increase customer/user acceptance of the application/system, by reducing customer

resistance to change [31].


- 87 -

4. To improve deliver a complete and accurate requirements [29].

5. To inform customer and understand any changes of the work processes and the

organisation, that caused by the system [29].

Agile - Common communication methods among team members



Project 7

0 1 2 3 4


Mostly formal


Mostly Informal


Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Figure 52: Agile – Common communication methods among team members

“One of the main reasons for using a software development methodology is to support and enhance

communication among stakeholders during all of the development phases” [32].

Following what have been mentioned above, figure 53 shows the result of the investigated projects

for the communication methods between team members and managers (refer to question 39 in the

questionnaire).

For some projects, the communication methods needs to be improved, if we take into account that

face-to-face communication is preferred than indirect communication, because it is more affective

and helpful.


- 88 -

Agile - Common communication methods between team members and managers

Project 1

Project 2

Project 3

Project 4

Project 5 Project 6

Project 7

Project 8

0 1 2 3 4


Mostly formal


Mostly Informal


Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Project 7

Project 8

Figure 53: Agile – Common communication methods between team members and managers

Figure 54 the results about leadership roles and experience on projects. Questions 40 and 41 in the

questionnaire are designed to collect data from participants of their roles, such as project

managers, team leaders, programmers, analysts, designers, architects etc.


development lifecycles. These roles are important and have an affect on project progress. Three

Questions (42, 43 and 44) in the questionnaire are designed to collect data from participants.


- 89 -

Agile - Project management/Leadership main roles on projects

0 1 2 3

1

2

3

4

5

6

7

8

Expe

rinec

e &

Pro

ject

No

Number of role(s)

Programmer

Designer Analyst

Tester

Team Leader

Business Analyst

Project Manager

Architect

Other

First time

2 years

2 years

6 years

10 years

10 years

1 year

First time

Figure 54: Agile – Project Management/Leadership main roles and experience

No evidence found to support the leadership roles. But, for some individual agile methods, I have

found some roles of agile methods illustrated at appendix I as stated by [5] and [6].

Agile - Project Management/Leadership experience

1

4

5

4

5

5

1

36

5

9

5

1

7

23

4

10

10

45

20

0 5 10 15 20 25 30 35 40 45 50

1

2

3

4

5

6

7

8

Proj

ect N

o

Number of years

Software development field experienceBusiness domain experienceSDLC experience

Figure 55: Agile – Project Management/Leadership experience


- 90 -

Figure 55 show management experience in software development field, business domain and

development lifecycles. All of the three roles are important to the project. The graph shows a very

straight forward result of participant’s lifecycle experience. The vast majority of participants have

average (5 years) experience in agile methods. The result also shows that, the majority of

participants have experience 5 years and above in software development field.

These improve and increase the validity of the research, that seven out of eight responses were

experienced participants.

I couldn’t found evidence to support these criteria.

Prototyping OO Structure Analysis & Design Pair Programming Other

Prototyping OO Coding techniques Pair Programming

Prototyping OO Coding techniques Structure Analysis & Design

Prototyping OO

Prototyping OO Coding techniques Structure Analysis & Design Pair Programming

Prototyping OO

Prototyping

Prototyping OO Coding techniques

1

2

3

4

5

6

7

8

Proj

ects

Agile - Techniques used in projects

Prototyping

OO

Coding techniques

Structure Analysis & Design

Cleanroom

Pair Programming

Other

Techniques

Figure 56: Agile – Techniques used in projects

Figure 56 shows the techniques used in the investigated projects. The techniques used in the

questionnaire are the common techniques used in industry. The result shows that, most of

participants agreed on the techniques that suitable with agile methods projects. Participant

responded with ‘other’, have used other techniques, which not stated in the questionnaire, the

techniques are, Story Points and Backlog.

This research is focusing on the development methods and techniques. Therefore, study of

individual techniques in depth is another area of study, which should be included in future study.


- 91 -

Agile - Length of iteration (weekly)

2

4

5

23

2

3

6

1

2

3

4

5

6

7

8

Project

Project

Project

Project

Project

Project

Project

Project

Figure 57: Agile – Length of iteration (weekly)

Figure 57 show the results of iteration length for agile methods used in investigated projects. The

longest iteration used at project 1 is 6 weeks, and the shortest iteration is 2 weeks, used at projects

3, 5 and 8.

The length of iteration of 2 weeks is acceptable, it allow controlling the development process. The

shorter iteration of 1 week, could be very short, and does not give much progress, as well as lose of

control. The 8 projects investigated above, shown a reasonable use of the iteration length.

The length of iteration assume to be set base on the nature of the project, the team size, project

complexity and the experience of the project team and leadership etc.

There is no evidence to support these criteria.

Agile method (ASD) is focused on iterations and constant prototyping and is basically supposed to

be used for the development of large software systems [5].

Concerning the advantages and disadvantages of agile methods used in the investigated projects,

two questions 48 and 49 has been set to collect data from participants about agile methods. The

results are shown in figure 57 and figure 58 below respectively.


- 92 -

Figure 58 illustrates the advantages of the agile methods used in the investigated projects. The

graph is very straightforward.

Description of the figures, the items A, B, C, D etc, represent the statements from questions 48 and

49 in the questionnaire, the graphs 59 and 60 represent the results of questions respectively. The

letter K reflect participant’s respond with ‘other’

Agile - Advantages/Benefits of the used lifecycles

A

A

A

A

A

B

B

B

B

B C

D

D

D E

F

F

F

G

G

G

G

G

G

H

H

H

H

H

H

H

I

I

I

I

I

J

J

J

K

K

0 1 2 3 4 5 6 7 8 9 10

1

2

3

4

5

6

7

8

Proj

ect N

o

Advantages

A

B

C

D

E

F

G

H

I

J

K

Figure 58: Agile – Advantages/benefits of the used lifecycles

Project 1, K = Limiting the accumulation of unknowns

Project 4, K = A focus on regularly delivering (or aiming to deliver) production quality software

that can be released and used every three 3 weeks.

For questions 48 and 49 refer to appendix J


- 93 -

Agile - Disadvantages/Limitations of the used lifecycles

A

A

A

B

B

C

C

C

E

E

E

E

G

G

G

G

G

G

H

H

I

I

I

J

K

K

K

0 1 2 3 4 5 6 7 8 9 10

1

2

3

4

5

6

7

8

Proj

ects

Disadvantages

A

B

C

D

E

F

G

H

I

J

K

Figure 59: Agile – Disadvantages/limitations of the used lifecycles

Project 1, K = New releases every six weeks makes it hard to control the number of bugs

Project 4, K = Not having enough analysis material in order to fully understand what is needed in each deliverable

Project 6, K = Poor software quality, time pressure, increased defects

Last question in the questionnaire is question 50, I have asked participants “under which condition

particular development methodologies should not be used”, and the responses collected from this

question are illustrated in the table 22 below (Appendix H).

Findings – Agile methods

My founding will be based on the software development lifecycle under investigation. And on the

light of the analysis carried out on agile methods, I will list what I have found as follow:

1. Agile methods have been used in small, medium and large organisations (see table 17).

2. Agile methods have been used with different projects sizes. It has been used in large, medium

as well as in small projects (see table 17)

3. Agile methods used in the investigated projects are: (XP, SCRUM, RUP, LSD)


- 94 -

4. Agile methods has been used in different types of projects, Internet, Intranet, Management

information system, Transaction processing, Decision support, Database, Data mining,

Document management, Communication and Sensor data fusion (see table 17).

5. The vast majority of the participated projects have used prototyping techniques.

6. The vast majority of participants used agile methods have high experience.

7. The newer the programming languages used in projects, the more likely agile methods to be

used (table 18)

8. Agile methods are suitable for high critical projects.

9. The experienced leadership in software development, the more likely to use agile methods

(figure 55)

10. Seven out of ten participants have responses (indicated) that the team is not always involved in

decision making

11. The well established requirements, leads to good quality product

12. I have found no evidence to support the ideas that the project factors such as size, time

pressure, complexity, quality, budget and technology affects the use of agile methods.

13. The results founded from advantages and disadvantages questions for the used of waterfall

lifecycle as follow (see figures 58 & 59), where participants agreed on them:

Advantages a. Five responses indicated that, agile methods have more control of the development process.

b. Five responses indicated that, agile methods increase productivity

c. Three responses indicated that, agile has less bugs/defects in final product

d. Six responses indicate that agile improve communication between team members

e. Seven responses indicate that agile methods improve communication with customers/clients

f. Five responses indicate that agile method shorter development time frame

Disadvantages a. Three responses out of eight indicated that agile methods have difficult to control requirements

changes

b. Two responses out of eight indicated that agile methods have difficulties to meet

client/customer requirements/needs


- 95 -

c. Three responses out of eight indicated that agile methods can’t prioritise requirements

d. Six responses out of eight indicated that agile methods have in controlling iterations

When to use and not to use Agile methods based on the research data

Use Waterfall Not to use Waterfall

Agile should also be used with caution. It

tends to emphasise the programming team

and de-emphasise the Architecture of the

system. This fails to recognise the role that

architecture has in change management.

(from project 1)

Agile probably wouldn't work so well in an

environment where the team wasn't

experienced / technically capable and the

management wasn't willing to support the

methodology. The team needs to be willing

to take on the challenge of learning to

follow agile but in doing so they need to

have enough technical skills to confidently

produce useful deliverables every iteration

with limited up-front analysis.

Additionally, if management doesn't

support the methodology then you risk

being stranded/isolated if an issue arises

that needs to be

(from project 4)

SCRUM - when strong management

support and team commitment is absent.

(from project 5)

When two sources delivering business

requirements and both can not agree.

(from project 7)

Table 23: Agile – When to use and not to use Agile methods

Finally, from the analysed results of the investigated projects on agile methods, and on the

light of the given data, the size of the projects intended to use agile methods should be

restricted on small to medium projects. There is no evidence to support this assumption.


- 96 -

Incremental lifecycle

Incremental is the third investigated method in my project. The tables and figures below, shows the results of the investigated projects that used incremental

lifecycle. Table 24 shows the number of participants that used incremental lifecycle, techniques,

project type, project size and organisation size. The results show that, all projects sizes are small

except one project.

The result shows that, six projects out of seven are using prototyping technique. Also all

techniques used with incremental lifecycle are techniques stated in the questionnaire.

Clearly the result shows that, there are some sorts of agreement on the use of techniques with

incremental lifecycle. The techniques stated in the questionnaire are common and the most likely

techniques currently in use in industry.


- 97 -



1 Coding techniques Intranet, Transaction processing, mainframe, Database

Don’t know

Don’t know

300 50

2 Prototyping, Structure analysis/design

Data mining 4 14 30 0

3 Prototyping, OO, Structure analysis/design

Intranet, Training system

5 2 6 1

4 Prototyping, Coding techniques


5 Prototyping, OO, Coding, Structure analysis/design, Pair programming

GUI application 1 4 30 0

6 Prototyping, OO Internet 60 0 25 0

Incremental

Projects

7 Prototyping Intranet, Data mining 5 8 9 1 Table 24: Incremental & Techniques (by project and organisation sizes)


- 98 -

Project 2

Project 3

Project 5

Project 6

Project 1

Project 2

Project 3

Project 4

Project 5

Project 5


Incremental - Determine completion of projects

Project 6

Project 5

Project 4

Project 3

Project 2

Project 1

Figure 60: Incremental – determine the completion of project


Project 5

Project 1

Project 6

Less than 5defects

Between 5 and20 bugs



Other

Incremental - Determine completion of projects by defects

Project 6

Project 5

Project 4

Project 3

Project 2

Project 1

Figure 61: Incremental – determine the completion of projects by defects


- 99 -

The results of the two figures stated above, concerning the quality of the investigated projects.

Refer to table 26 below, the results shows that, only projects 3 and 5 were completed and the rest

are underway projects, and the duration of both projects are 2 years and 8 years respectively.

Concerning the quality of the produced product (system/application), three questions (16, 32 and

33) in the questionnaire have been set to collect data from participants in order to measure and

determine product quality. The questions prompt participants to provide data about, completion of

project (e.g. project signoff, project in production), by the number of defects in the product and by

indicating product quality by using other factors.

These questions should give strong evidence if incremental will fulfil customer’s

needs/requirements and if customer satisfaction achieved.

All projects have duration over one year, except one project with duration of four months.

Figure 60 show that, the majority of the projects have chosen two methods to determine the

completion of projects, in order to get the highest possible quality in the final product, by meeting

customer’s needs/requirements and technically the product is in production.

Project 5 used three methods, the other method given by participants is ‘Maintenance phase reduce

pace of development’, but I can not see any relation between the determination of project and

maintenance, which is the stage after the release of the product, except that, the delivery of high

quality product will reduce future maintenance.

One participant (project 1) selected one method only in determining the completion of the project,

when project is in testing.


Figure 61 show the quality in the expected product, result show that quality is not high for all

projects.

If we look to the two completed projects 3 and 5, both projects have determined the completion as,

project in production and customer signoff, with a number of defects between 20 and 50 defects

for both.


- 100 -

We back to question 32, this question directly asking participants about product’s quality, as stated

in figure 65. The result show that project 3 completed with low quality and project 5 completed

with high quality.

Therefore, from the results we have, we have slight evidence, that:

• Product in production and customer signoff are successful methods to determine the

completion of the project and to meet customer satisfaction.

• Incremental dose not (or hardly) produce product free of defects.

• Incremental is mostly suitable for big projects and can fulfil customer’s

requirements/needs.

Project complexity Concerning complexity in the investigated projects, two questions (23 and 24) in the questionnaire

have been set, asking participants to provide data about complexity.

Table 25 below show the results of question 23. The complexity can be measured or determined

based on different factors. I have used two ways. First, asking participants about the tools used in

the projects such as, programming languages, technology, testing tools, management tools, design

etc. Second, by asking participants directly about the rate of complexity in the project using other

factors involved in project development.

The data shown in the table are exactly as given by participants.

Management tools MS Project, MS Excel

Automation testing tools Test Complete

Case Modelling tools BridgePoint, Enterprise Architect

Testing tools AQTime

Programming language C++, C#


Open source tools

Source control tools SVN

Project 1

Other In house developed Quality tools - Requirements tracing


- 101 -

Management tools



Testing tools

Programming language C, C++

Design tools

Open source tools Boost C++ template libraries, QT GUI libraries


Project 2

Other In house build and packaging tools written in lua. Use of Incredibuild network build tool.

Management tools



Testing tools JUnit and JUnit utilities (DBUnit, Emma, etc)

Programming language Java, JavaScript/HTML, Flash.

Design tools Visio

Open source tools Eclipse and the whole suite of Eclipse products


Project 3

Other Archiva, Continuum

Management tools Project

Automation testing tools Rational Rose


Testing tools

Programming language C, C++, C#, PL-SQL

Design tools

Open source tools Git

Source control tools Git

Project 4

Other


- 102 -

Management tools Microsoft

Automation testing tools TFS build


Testing tools Hudson

Programming language C#,T-SQL


Open source tools

Source control tools TFS

Project 5

Other ORM

Management tools MS Excel

Automation testing tools PyTest

Case Modelling tools MS Word

Testing tools PyTest

Programming language C++,C#,Python

Design tools

Open source tools Open Office, Doxygen


Project 6

Other SOAP Table 25: Incremental - Project complexity first criteria

The complexity can not be directly determined, based on factors such as novelty of the technology,

the novelty of the business problem, team skills and the combination of technologies used [35].

Based on that, table 25 above shows the variety of complexity used within the projects. In terms of

the programming languages used in projects, the table show Java, JavaScript, C#, C++, C, HTML,

Flash, PL-SQL, T-SQL, Python.

The programming languages used in the projects, are common programming languages. The

languages do not lead to high complexities in projects.



- 103 -

C

D

E

F

A

B

C

E

F

A

C

D

A

B

C

D

E

D

E

A

B

C

1 2 3 4 5 6Projects

Incremental - Complexity of Projects

F

E

D

C

B

A

ABCDEF

Figure 62: Incremental – Project complexity second criteria

Figure 62 illustrates the results from question 24. It shows the level of complexity of each project.

In figure 62 the colours of the bars reflects/indicates levels of complexity on projects, question 24

is attached to the graph (question 24 in questionnaire).

Graph description: the letters (A, B, C, etc) indicates the level of complexity, which reflects the

statement selected in answering the question, which is corresponding the marked statements in the

question.


Rate of change in the projects and constraints Defining the status of the received requirements at the beginning of the projects, and rate requests

of changes received are the criteria used to define the rate of changes in projects. Five questions

(25, 26, 27, 28 and 29) set in the questionnaire to collect data from participants. The results of

these questions have been presented in figures 63 and 64.


- 104 -

1 2 3 4 5 6Projects

Incremental - Project requirements defined at the beginning of the project (including user andbusiness requirements)

Complete

Ambiguous

Partial

Incomplete

Figure 63: Incremental – Projects requirements defined at the beginning of the project

Figure 63 is the results of the collected responses about the defined project requirements at the

beginning of the project (question 25 – questionnaire).

The coloured bars reflect requirements status at the beginning of the project for the investigated

projects. The results of these questions with correlation with the other questions stated above,

should help to specify the rate of changes in the projects.

The results show that none of the projects have received complete requirements at the beginning of

the project, with correlation to figure 64, projects 2 and 3, have received incomplete requirements,

as well as neither fixed deadline nor budget has been set, project 2, and project 3 has ongoing

(constant) changes, such changes put the development team under pressure and in unstable project

environment (by the time the team deliver part of the system “or whole”, the requirements will be

changed).

Does the lifecycle (or the team) can cope with such project environment?



- 105 -

Proceeding with the analysis, table 12 (Appendix J), shows the questions that addressing the rate of

changes in the projects, except for question 25 (from questionnaire). Questions 26-29 will help

translating the data presented at figure 64 below.

1 2 3 4 5 6

? 550

0

onging

1 0 2200

10+45 50+

10054

10+

25 >25

200187

10+

Projects


Question 26

Question 27

Question 28

Question 29

No staff changes. No fixed deadline or budget has been set.

Figure 64: Incremental – Rate of change in projects by request

Figure 64 shows four rows and six columns, the column number refer to the participated project,

every row has a unique colour which reflects one question of the four questions mentioned above.

The numbers on the top of the bars, indicates the number for requests of changes received.

The number with a plus (such as 10+), indicates that the number of requests received by any of the

four questions is in tens, which mean, participants did not response with exact number, instead

they respond with ‘tens’ of requests. Similar for (>25), participants indicates that the requests of

changes received are greater than 25, such as for project 6.



- 106 -

Incremental - Level of quaity in the projects

1

1

1

1

1

0

1

2

3

4

5

6Very high quality

High quality

Good quality

Low quality

No quality

Figure 65: Incremental – Level of quality expected in the projects

Figure 65 show the level of quality in projects, the colours and the numbers at the top of the axes

refer to the project number. At the centre of the chart the quality starts at zero (no quality) and then

it increase until it reach to the highest marks in the axes, where the indicators reach to the value of

very high quality (such as for project 4).

It is assumed that incremental produce high to good quality product.

It is clearly that, the demand on very high project quality does not prevent or reject the use of

incremental in the project development. It is very common and in many cases, that software

development projects do not deliver the quality of the product as intended or required by the

customer. “Customer, often complain that their real requirements are hardly met and that the

software is too difficult to use” [27].

As it can be seen in the graph, the majority of participants reported on projects, did not expect high

quality on their product, therefore, incremental in selected for their projects. Is this because

incremental did not deliver high quality product?



- 107 -



1 NO 2 years

2 NO 4 months

3 YES 2 years

4 NO 1 year

5 YES 8 years

6 NO 1 year Table 27: Incremental - Investigated project status and duration

Table 27 shows the status of the investigated incremental projects and its durations, by project

number.

Project criticality




Nothing much Other

1

The project is more about understanding the technology, and less about delivering a product. The primary purpose is to provide skills training, and the secondary purpose is to provide a product (or something that may become a product). A likely risk factor is if the secondary purpose becomes the primary, then damage to reputation is possible.

2 Yes 3 Yes 4 Yes 5 Yes Potential legal issues 6 Yes

Table 28: Incremental – How critical is the failure of project to the organisation

Table 28 shows how critical is the failure of the project to the organisation. We can see that project

4 is very critical to the organisation and the consequences of projects failure are potential loss of

human life. Found that, incremental lifecycle is suitable for low and high critical projects.

No other evidence found to support these criteria.


- 108 -

Incremental - Projects budget

Project 1

Project 2


Project 6

0 1 2 3 4

Generous budget

Sufficient budget

Insufficient budget

No budget

Don't know

Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Figure 66: Incremental – Projects budget

Figure 66 represent the result of projects budgets. The data has been collected from the responses

on question 30.

The important of these data are indirectly reflects the important of the project to the organisation as

well as if the budget adequate to achieve business requirements and will help to produced quality

product. The graph shows the budget status of the projects as well as the project name at each level

(such as project 1, project 2 etc).

Project budget has an influence on the development lifecycles and on final product.

Lets have a look at projects 4, 2 and 6, project 4 is very critical project as we have seen before, but

at the same time has insufficient budget, projects 2 and 9 has no budgets. Project 6 has no expected

quality, which indicates that the projects have no values to the organisation.

No evidence found to support these criteria. But, it is obvious project with insufficient budget will

have influence on all project factors including quality (as in project 4).


- 109 -

Team experience and communication

Incremental - Team experience

Project 1

Project 2 Project 3

Project 4

Project 5

Project 6

0 1 2 3 4


High experience

Good experience

Little experience


Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Figure 67: Incremental – Project team’s experience

Figure 67 shows the level of experience of the project’s team. Question 35 in the questionnaire has

been set to collect responses for this purpose.

In general, the more experienced is the team, the more successful of using software development

methodologies.

The result shows clearly that, none of the teams has very high experience, but the average

experience of the teams is on high level. At the same time it can be seen as for projects 2 and 3,

both with little experience.

Does the importance of the project have an influence on team structure (from experience

prospective)?



- 110 -


participants to measure the level of team’s involvement in decision making in projects.

Incremental - Team empowerment on the projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

0 1 2 3 4

Always

Often

Rarely

Sometimes

Never

Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Figure 68: Incremental – Team empowerment

Figure 68 shows the result of the collected data from question 36 in the questionnaire. The graph

illustrates the team involvement level in each project. The colours in the legend are to connect

projects to their corresponding level of involvement by project.

From the analysed results shown, only project 1 have respond ‘always’, which mean the team

always involved in decision making when there are problems in the project.

Projects 2, 4 and 6 responded with ‘often’, and the rest responded with ‘sometimes’.

Does incremental support team empowerment?



- 111 -

0 1 2


1

2

3

4

5

6

Proj

ects

Incremental - Team meetings (fortnightly)

Figure 69: Incremental – Frequent team meeting (fortnightly)

Figure 69 shows the result of teams meetings fortnightly. For example project 1 team conduct two

meetings fortnightly, frequent (reasonably) team meeting is very important to the project.

The needs for frequently team meetings, such as once a week, is necessarily, specially when the

project is too complex, requirements are not well defined, project not progressing well, when team

members are in needs to share knowledge and experience and when risk issues occurs should be

discussed.

The result show, projects 2 and 3 has little experience, one of the ways to improve team’s

knowledge and share skills among team members is to conduct regular team meeting to discuss the

issue deeply.

Refer to the benefit of team meetings illustrated in the analysis of agile methods above.

I’ve found no evidence to support these criteria


- 112 -

Incremental - Common communication methods among team members

0 1 2 3 4


Mostly formal


Mostly Informal


Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Figure 70: Incremental – Common communication methods among team members

Figure 70 shows the results of communication methods used between team members (question 38

in the questionnaire) for all investigated projects. Team communication and meetings help the

development team to use the most effective way to achieve project goal (if used correctly and

effectively).

The result show that, only project 1 is using mixture of formal and informal communication

methods, the rest of the teams using either formal or informal communication methods.

No evidence found supports these criteria.


- 113 -

Incremental - Common communication methods between team members and managers

0 1 2 3 4


Mostly formal

Mixture of formal andinformal

Mostly Informal


Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

Figure 71: Incremental – Common communication methods between team members and managers

Continue from the analysis above, figure 71 shows the results of investigated projects on

communication methods between team members and managers (question 39 in the questionnaire).

Project 2 used informal communication method, and other projects used mixture and mostly

formal communication methods.

No evidence found to support this criterion.


- 114 -

Incremental - Project management/Leadership main roles on projects

0 1 2 3 4

1

2

3

4

5

6

Expe

rinec

e &

Pro

ject

No

Number of roles

Programmer

Designer Analyst

Tester

Team Leader

Business Analyst

Project Manager

Architect

Other

2 years

6 years

2 years

8 years

6 years

7 years

Figure 72: Incremental – Project Management/Leadership main roles and experience

Figures 72 and 73 showing data and results of statistical analysis from participants’ responses on

their roles and experience on projects, respectively. Five questions (40, 41, 42, 43 and 44 in the

questionnaire) are designed to collect data from participants about their roles and experience, such

as project managers, team leaders, programmers, analysts, designers, architects etc.

No evidence found supports these criteria.


- 115 -

Incremental - Project Management/Leadership experience

2

4

4

5

2

8

1

5

4

1

2

4

6

5

12

7

5

12

0 2 4 6 8 10 12 14

1

2

3

4

5

6

Proj

ects

Number of years



SDLC experience

Figure 73: Incremental – Project Management/Leadership experience

Figure 73 show management/leadership experience in software development field, business

domain and software development lifecycle (SDLC) experience. All of the three roles are

important to projects. The graph shows very straight forward results to all roles. The majority of

participants have four years and more experience in SDLC. The result also shows the majority of

participants have five years and more experience in software development.

These improve and increase the validity of the research, participants with high experience in

software development, are more likely to have more experience in software development lifecycles

(SDLC).



- 116 -

0 1 2 3 4

Techniques

1

2

3

4

5

6

Proj

ects

Incremental - Techniques used in projects

Prototyping

OO

Coding techniques


Cleanroom

Pair Programming

Other

Figure 74: Incremental – Techniques used in projects

Figure 74 shows the techniques used by the investigated projects. The techniques used in the

questionnaire are the common techniques used in industry. The result shows, participants have

agreed with the used techniques that are suitable for use with incremental. Participants responded

with ‘other’, mean they have used other lifecycle, such as project 6 “Used our own organisation

development lifecycle”.


Now, concerning the advantages and disadvantages of incremental lifecycle used in the

investigated projects, two questions has been set to collect data from participants, questions 48 and

49, the results are shown in figure 75 and figure 76 respectively.

Figure 75 illustrates the advantages of the incremental lifecycle used in the investigated projects.

The graph is very straightforward.

Description: the letters A, B, C, D etc in the graph, represents the statements in the questions (48

and 49), which is corresponding to participant’s response on questions 48 and 49. The letter K

replace response ‘other’ to match with the correspond statement.


- 117 -

Incremental - Advantages/Benefits of the used lifecycles

0 1 2 3 4 5 6

1

2

3

4

5

6

Proj

ects

Advantages

A

B

C

D

E

F

G

H

I

J

K

Figure 75: Incremental – Advantages/benefits of the used lifecycles

Project 1, K = Reduced risk of developing the wrong/unnecessary functions.

Project 6, K = Increased freedom for developers.

Incremental - Disadvantages/Limitations of the used lifecycles

B

E

0 1 2 3 4 5 6 7 8 9 10

1

2

3

4

5

6

Proj

ects

Disadvantages

ABCDEFGHIJK

Figure 76: Incremental – Disadvantages/limitations of the used lifecycles


- 118 -

Project 1, K = Within the team, each engineer is responsible for a CSCI, from requirements,

design, code and test. The CSCI's cannot be built in parallel, as functions from one are used to

enable the next. The integration of the various components becomes the problem, as each CSCI is

in a different development phase.

Project 5, K = Poor software quality. Time pressure. Increased defects

For questions 48 and 49 refer to Appendix J


particular development methodologies should not be used”. The responses collected from this

question are illustrated in the table 29 (Appendix H).

Note: Participants number 3 and 5 did not provided answer to question 50.

Findings – Incremental lifecycle

My founding will be based on the software development lifecycle under investigation, and on the

light of the analysis carried out on incremental lifecycle, the list of founds are:

1. I’ve found that the incremental have been used in small, medium and large projects, as well as

in small, medium and large organisations (see table 24).

2. Incremental lifecycle has been used for different types of projects, for Internet, Intranet,

transaction processing, mainframe, database, data mining, training system, communication and

GUI application (see table 24).

3. The vast majority of the participated projects have used prototyping techniques.

4. The vast majority of participants used incremental have high experience.

5. The programming languages used are very similar to the ones used in agile methods

6. Incremental lifecycle is suitable for low and high critical projects (table 28).

7. Five out of six teams have always involved in decision making (empowerment on the projects)

8. None of the participants receiving complete requirements at the beginning of the projects.


- 119 -

9. I have found no evidence to support the assumption that project factors such as size, time

pressure, complexity, quality, budget and technology affects the use of incremental lifecycle.

10. The advantages and disadvantages of incremental lifecycle as follow

Advantages 1. Four participants only responded to the question, two responded with ‘other’

2. Two participants agreed on the following:

1.1 Incremental has control on the development process

1.2 Incremental improved communication between team members

1.3 Incremental improved communication between team members Disadvantages 1. Five responses out of six responded to the question

2. More responses on disadvantages than on advantages

3. Five responses indicated that, controlling iterations is difficult

4. Four responses indicated that, delays to delivery of product

5. Four responses indicated that, difficult to meet client/customer requirements/needs

6. Three responses indicated that, poor design documentation

When to use and not to use incremental based on the research

Use Incremental Not to use Incremental

Use incremental if you know how to

implement it and apply process

(from project 1)

Do not use it if you don’t know how to

implement it

Use the methodology if you really have a

faith with it, and you certain that it will work

well with your team and your project

Use the methodology if you think it will

handle changes in requirements probably

When two sources delivering business

requirements and both can not agree.

Table 30: Incremental – When to use and not to use incremental methods


- 120 -

Iterative lifecycle

Iterative is the fourth investigated method in the report.

The tables and figures below, shows the results of the investigated projects that used iterative.

Table 31 shows the number of participants that used iterative, techniques, project type, project size

and organisation size. The results show that, the projects used iterative are small to medium in

sizes, by small, medium and large organisations.

The result also shows, eight out of nine projects used Structure analysis/design technique, and six

out of nine are using prototyping technique. Also all the techniques used with iterative are used in

the questionnaire.

Clearly the result show, there are some sorts of agreement among participants on the use of

techniques with iterative. The techniques used by projects as well as the questionnaire, are

common and the most techniques currently used in industry.


- 121 -



1 Structure analysis/design

Intranet, Transaction processing, mainframe, Database

10 1


Intranet 4 2 30 10

3 Prototyping, Coding techniques, Structure analysis/design

7 2 0 8


Management information system, Decision support

30 0 40 0


Internet, Intranet, Database

30 10 2500 500

6 Prototyping Internet, Intranet, Management information system, Database

15 4 10000 1000


Transaction processing, Data mining

10 0 6 0

Iterative

Projects

8 Prototyping, OO, Structure

Intranet 25 10 450 80


- 122 -

analysis/design 9 Prototyping, OO,

Coding techniques, Structure analysis/design

Internet, Transaction processing

4 1 285 80

Table 31: Iterative & Techniques (by project and organisation sizes)


- 123 -


Iterative - Determine completion of projects

Project 5

Project 4

Project 3

Project 2

Project 1

Figure 77: Iterative – determine the completion of project

Figure 77 show the result of the methods used by participants in order to determine the completion

of projects, the colours refer to projects as in legend. The figure shows that, none of the

participants have chosen both methods, one project response with ‘other’.

The results of the two figures (77 and 78), concerning the methods used to determine the

completion of projects. Table 32 below show, projects 3 and 5 were completed projects and the

rest are underway, project 3 was 18 months duration and project 5 is not given. In additional to the

above, quality is also a concern, 3 questions (16, 32 and 33) in the questionnaire have been set to

collect data from participants, in order to measure and determine the project completion (e.g.

customer signoff, project in production etc), product quality by asking participants directly and

number of defects in the product, respectively.

The duration of the projects varies from 6 months to 3 years.


- 124 -

Less than 5defects

Between 5and 20 bugs



Other

Iterative - Determine completion of projects by defects

Project 5

Project 4

Project 3

Project 2

Project 1

Figure 78: Iterative – determine the completion of projects by defects

Figure 78 show that, none of the projects are free of defects. Project 4 went far by showing the

number defects in the project are between 50 and 100, with correlation to figure 82, which

represent the result of question 32 (concerning product quality), the quality of project 4 point to

“very high”. It is clear there is no match between the expected quality and number of defects in

this project.

If we look to the two completed projects, 3 and 5, it can be seen that both projects have determined

the completion by customer signoff and project in production respectively, with a number of

defects between 20 and 50 bugs and between 5 and 20 bugs respectively.

Therefore, from the results we have slight evidence:

• ‘Product in production’ and ‘customer signoff ‘are successful methods if both are chosen/used.

• Iterative dose not (or hardly) produce product free of defects.

• Incremental is mostly suitable for small to big projects and can fulfil customer’s requirements.

• Iterative almost suitable with all techniques used in the questionnaire.



- 125 -

Project complexity

Concerning complexity with iterative, 2 questions (23 and 24) from questionnaire have been used

for this purpose.

Table 32 below show the results collected from question 23. The question set to collect

information from participants about the complexity of the projects.

The complexity can be measured or determined based on different factors. First, asking

participants about some project factors such as, programming languages, technology, testing tools,

management tools, design etc. Second, by asking participants direct question (question 24) about

the rate of complexity in the project using other project factors involved in the project

development.

The data shown in table are exactly as given by participants.

Management tools Microsoft Project / Excel

Automation testing tools In-house development (Oracle PL/SQL based)

Case Modelling tools Oracle Designer

Testing tools In-house development (Oracle PL/SQL based)

Programming language SAS, Oracle PL/SQL, Java

Design tools Oracle Designer

Open source tools


Project 1

Other IBM Cognos 8.4

Management tools



Testing tools

Programming language Java

Design tools

Open source tools Spring evamework

Project 2

Source control tools IBM Rational Clearcase


- 126 -

Other


Automation testing tools Mercury QC


Testing tools

Programming language Java, XML/XSL, JavaScript, etc

Open source tools

Source control tools CVS, ClearCase

Project 3

Other

Management tools



Testing tools

Programming language Cobol,MS.NET

Design tools

Open source tools


Project 4

Other


Automation testing tools Junit Testing

Case Modelling tools UML

Testing tools

Programming language C++,C#,Python

Design tools

Open source tools Netbeans IDE

Source control tools CVS

Project 5

Other Table 32: Iterative - Project complexity first criteria


- 127 -

The complexity can not be directly determined, it is based on factors such as novelty of the

technology, the novelty of the business problem, team skills and the combination of technologies

used [35].

Based on that, table 32 above shows the variety of factors used in projects that reflect complexity.

In terms of programming language used in the projects, the table show projects have used: (SAS,

Oracle PL/SQL, Java, XML/XSL, JavaScript, COBOL, MS.NET, C++, C#, Python).

The programming languages used in the projects, are common programming languages, also they

varies from new to old programming languages.

No evidence found that relate programming languages to neither iterative nor to the complexity.

1 2 3 4 5

Projects

Iterative - Complexity of Projects

F

E

D

C

B

A

ABCDEF

Figure 79: Iterative – Project complexity second criteria

Figure 79 below illustrates the results from question 24. It shows the level of complexity of each project. The colour in figure 79 reflects the levels of complexity in projects, which are collected from

question 24, question attached to the graph (question 24 in the questionnaire).


- 128 -


statement selected in answering the question.

The graph shows that five out of six projects, responded with option E “some team members are

unfamiliar with iterative”, also four projects responded with option A “integration of more than

two applications”.

I couldn’t find any evidence that support these criteria. Rate of change in the projects and constraints Concerning the status of the received requirements at the beginning of the projects, and the rate for

requests of changes received are the criteria used to define the rate of changes in projects. Five

questions (25, 26, 27, 28 and 29) set in the questionnaire to collect data from participants. The

results of these questions have been presented in figures 80 and 81.

Proceeding with the analysis, table 12 (Appendix J), shows the questions that addressing the rate of

changes in the projects, except for question 25. Questions 26-29 will help translating the data

presented at figure 81 below.

1 2 3 4 5

Projects

Iterative - Project requirements defined at the beginning of the project (including user and business requirements)

Complete

Ambiguous

Partial

Incomplete

Figure 80: Iterative – Projects requirements defined at the beginning of the project


- 129 -

Figure 80 illustrate the results of the collected responses about the requirements status at the

beginning of the projects (question 25 – questionnaire). The coloured bars reflect requirements

status (in the legend) at the beginning of the project. The result of this question with correlation to

the result of questions 26-29, should help to specify the rate of changes in the projects.

The results show, only one participant (project 5) have received complete requirements at the

beginning of the project, if we correlate this to figure 79, we see that project 5 has five statements

selected. The rest of participants received incomplete requirements.


1 2 3 4 5

15 30 0310

2 0 080

20+ 10 5

1000+

20

100+ 50 51000+

12

Projects


Question 26

Question 27

Question 28

Question 29

30 Changes to budget and delivery

Figure 81: Iterative – Rate of change in projects by request

Figure 81 shows 4 rows and 5 columns for each project, every row has a unique colour which

reflects one question of the 4 questions mentioned above.

The numbers on the top of the bars, indicates the number of requests received.

The number with a plus such as (100+), indicates that the number of requests received is in

hundreds, which mean, participants did not response with exact number, instead responded with

‘hundreds’ of requests, and for (1000+), indicates number of requests received are in thousands, as

in project 4. Project 5, this project is stable and received 10 requests in total. No evidence found to

support these criteria.


- 130 -

Iterative - Levelof quaity expected in the project

1

11

1

1

1

2

34

5Very high quality

High quality

Good quality

Low quality

No quality

Figure 82: Iterative – Level of quality expected in the projects

Figure 82 show projects quality, the colours and the numbers at the top of the axes refer to project

number. At the centre of the chart the quality starts at zero (no quality) and then it increase till

reach to the highest mark in axes, where the indicators reach to the value of very high quality (such

as projects 2 and 4).

The result shows for the completed projects 3 and 5 are good and high quality respectively.




1 NO 3 years

2 NO 1 year

3 YES 18 months

4 NO 6 years

5 YES Table 83: Iterative - Investigated project status and duration

Table 33 shows the status of the investigated iterative projects and its durations, by project number.


- 131 -

Project criticality

Table 34 present data about criticality in projects, the data collected from participants’ responses

on question 34. It show how critical is the failure of project to the organisation. We can see that all

of the projects are critical to the organisation and could cause damage to the organisation

reputation. This criteria reflects criticality

Found that, iterative lifecycle is more suitable for high critical projects.





Nothing much Other

1 Yes

Yes

Budget/Planning/Senate Estimates incorrect - decisions made on incorrect information

2 Yes 3 Yes 4 Yes Yes 5 Yes

Table 34: Iterative – How critical is the failure of project to the organisation

Iterative - Projects budget

0 1 2 3

Generous budget

Sufficient budget

Insufficient budget

No budget

Don't know

Number of projects

Project 1

Poject 2

Project 3

Project 4

Project 5

Figure 83: Iterative – Projects budget


- 132 -

Figure 83 present the result of projects budget. This factor reflects criticality in projects, the data

has been collected from participants’ responses on question 30.

The important of these data are indirectly reflects the important of the project to the organisation as

well as if the budget adequate to achieve business requirements and will help to produced quality

product. The graph shows the budget status of projects, the colours refer to projects in the legend.

Project budget has an influence on other factors such as development lifecycles, requirements

gathering, design, testing, project quality etc.

Lets have a look at projects 3 and 5 and make a comparison, project 3 completed with insufficient

budget, and project 5 completed with sufficient budget, but quality of project 3 has been affected

as been given “good quality”, project 5 completed with better quality “High quality”. Project 2,

indicated no budget, so this project has started with no budget (something does not apply in real

life), this is lead us to ask, is this project has no value to the organisation?

I couldn’t found evidence that support the influence of the project budget on iterative. But, it is

obvious that projects with no budget will go no where (no progress), I believe the only work can be

done on this project is limited on some documentation. But, we must keep in mind that, reduce or

not produce design documentation, definitely will create a problem.



- 133 -


Iterative - Team experience

0 1 2 3 4


High experience

Good experience

Little experience


Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Figure 84: Iterative – Project team’s experience

Figure 84 shows the level of experience of the teams.

Question 35 (in questionnaire) has been set to collect data from participants’ responses.

It is valid to say, the more experienced is the team, the more successful of using software

development methodologies, until the opposite proven.

The result clearly shows, none of the teams has very high experience, but at the same time, the

teams have experience above the average.

The same as before I couldn’t found evidence to support these criteria.


participants to measure the level of teams’ involvement in decision making.


- 134 -

Iterative - Team empowerment on the projects

0 1 2 3

Always

Often

Rarely

Sometimes

Never

Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Figure 85: Iterative – Team empowerment

Figure 85 shows the result of the collected data from question 36 in the questionnaire. The graph

illustrates the number of projects at each level. The colours in the legend are to connect projects to

their corresponding level of involvement (e.g. project 1, project 2 etc).

From the analysed results, only project 4 have respond ‘always’, which mean the team always

involved in decision making when there are problems in the project.

Project 2 and 3 responded with ‘never’ and ‘sometimes’ respectively, and the rest responded

‘often’.

Does iterative support team empowerment?



- 135 -

0 1 2


1

2

3

4

5

Proj

ects

Iterative - Team meetings (fortnightly)

Rare - a major issue with this project. The instrument scientist(customer) was rearely available/engaged. About every 6

Figure 86: Iterative – Frequent team meeting (fortnightly)

Figure 86 shows the result of teams meetings fortnightly. For example, project 1 conduct two

meetings fortnightly, this is also apply on project 4.

The text placed at project 5 is participant respond on the question, the team complain about the

availability of the customer. I have no doubt to ensure, such a case has nothing to do with

methodology in use, such situation definitely will affect the project, the delivery date, and could

lead to freeze the project for a period of time, until the requested information is available.

Therefore, the affect of team meeting (between project stakeholders, team members/management)

has serious consequences on the project. This part also proves previous discussion about customer

involvement in the team.

I repeat that, the needs for frequently team meetings, such as twice a week, is necessarily, specially

when the project is too complex, requirements are not well defined, project not progressing well,

when the team members in needs to share knowledge and experience and when there is a need to

address risks and uncertainty that could occur on the projects.

Refer to the benefit of team meetings and customer involvement mentioned before.

Again, I’ve found no evidence to support these criteria


- 136 -

Iterative - Common communication methods among team members

0 1 2 3 4 5


Mostly formal


Mostly Informal


Number of projects

Project 1

Project 2

Project 3

Project 4

Project 5

Figure 87: Iterative – Common communication methods among team members

Figure 87 shows the results about communication methods used between team members. Question

38 in the questionnaire has been set to achieve this purpose. The graph shows the communication

methods used by all projects. Team communication, meetings and customer involvement helps the

development team to have effective strategy and techniques in order to achieve project goals

(include meeting customer’s requirements).

The result show that, only project 2 is using formal communication method. The other four

projects using mixture of formal and informal communication methods.

Now I can say without doubt that project 2 either has no experience in development methodologies

or participant’s role does not act as a leadership or at managerial level on the project, figure 87

shows that participant of project 2 are programmer and architect.



- 137 -

Iterative - Common communication methods between team members and managers

0 1 2 3 4


Mostly formal

Mixture of formal andinformal

Mostly Informal


Number of Projects

Project 1

Project 2

Project 3

Project 4

Project 5

Figure 88: Iterative – Common communication methods between team members and managers

Continue with the analysis from previous graph. Figure 88 shows the result of the investigated

projects for the communication methods between team members and managers (question 39 in the

questionnaire).

Project 1 have moved from mixture (in previous graph) to mostly formal communication method,

the question raise, dose this move help the team?

Other projects used mixture and formal communication methods, except project 2.

No evidence found to support this criterion.

Figure 89 below showing data and results of leadership’s roles and experience on projects.

Two questions (40 and 41) in the questionnaire are designed to collect data from participants about

their roles and experience, such as project managers, team leaders, programmers, analysts,

designers, architects etc. Project 1 indicates that leadership’s roles are, programmer, designer, team

leader, business analyst and architect, but the experience on these roles shown as “first time”,

which mean, this is the time acting on these roles.



- 138 -

Iterative - Project management/Leadership main roles on projects

0 1 2 3 4 5 6

1

2

3

4

5

Expe

rinec

e &

Pro

ject

No

Number of roles

Programmer

Designer Analyst

Tester

Team Leader

Business Analyst

Project Manager

Architect

Other

10 years

10 years

15 years

First time

1 year

Figure 89: Iterative – Project Management/Leadership main roles and experience

Iterative - Project Management/Leadership experience

10

5

7

1

3

20

5

1

2

5

32

20

10

3

8

0 5 10 15 20 25 30 35

1

2

3

4

5

Proj

ects

Number of years



SDLC experience

Figure 90: Iterative – Project Management/Leadership experience


- 139 -


development lifecycles. All of the three roles are important to the project.

Three questions (42, 43 and 44) in the questionnaire have been set to collect data from participants

to determine leadership experience on these roles.

The graph shows very straight forward results, for the software development lifecycle (SDLC)

experience, three out of five participants have five years and above of experience. The results also

shows that, four out of five participants have eight years and above of experience in software

development field, and three out of five participants have five years and above of experience in

business domain.



questionnaire are the common techniques used in industry. The result shows that, most of the

projects are agreed on the techniques that suitable with iterative projects. The project respond with

‘other’, have indicate “No real techniques other than prototyping”.

No evidence found to support these criteria for iterative.

0 1 2 3 4

Techniques

1

2

3

4

5

Proj

ects

Iterative - Techniques used in projects

Prototyping

OO

Coding techniques

Structure Analysis &Design

Cleanroom

Pair Programming

Other

Figure 91: Iterative – Techniques used in projects


- 140 -

Iterative - Advantages/Benefits of the used lifecycles

0 1 2 3 4 5 6 7

1

2

3

4

5

6

Proj

ects

Advantages

A

B

C

D

E

F

G

H

I

J

K

Figure 92: Iterative – Advantages/benefits of the used lifecycles

Now, concerning the advantages and disadvantages of iterative lifecycle used in the investigated

projects, two questions has been set to collect data from participants, questions 48 and 49, the

results are shown in figures 92 and 93 respectively.

Figure 92 illustrates the advantages of iterative used in the investigated projects. The graph is very

straightforward.

Description: the letters A, B, C, D etc, represent the statements in questions 48 and 49 (Appendix

J). The letter K represents respond ‘other’


- 141 -

Iterative - Disadvantages/Limitations of the used lifecycles

0 1 2 3 4 5 6 7 8 9 10

1

2

3

4

5

Proj

ects

Disadvantages

A

B

C

D

E

F

G

H

I

J

K

Figure 93: Iterative – Disadvantages/limitations of the used lifecycles


particular development methodologies should not be used”. The responses collected from this

question are illustrated in the table 35 (Appendix H).

The responses to question 50 is shown below as it answered by participants (participants

responses), the exact answers by projects:

Note: Participants 3, 4 and 5 did not provided answer to question 50.

Findings – Iterative lifecycle

My founding will be based on the software development lifecycle under investigation, and on the

light of the analysis carried out on iterative lifecycle, the list of founds are:

1. Iterative have been used in small, medium and large projects, as well as in small, medium and large organisations (see table 24).

2. Iterative has been used in different types of projects, (see table 30).

3. All participated projects have used the techniques used in the questionnaire.

4. The majority of projects teams have high experience

5. Projects have used both new and old programming languages (table 32)


- 142 -

6. Iterative lifecycle is more suitable for high critical projects (table 34).

7. I have found no evidence to support the assumption that project factors such as size, time pressure, complexity, quality, budget and technology affects the use of iterative.

8. The advantages and disadvantages of iterative as follow (figures 92 & 93): Advantages a. Three out of five responses indicated that iterative has more control of the development

process

b. Four out of five responses indicated that iterative deliver good quality product

c. Three out of 5 responses indicated that iterative has less bugs/defects in final product.

Disadvantages a. One participant has no response on disadvantages

b. Four responses indicated that, requirements changes are difficult to control

c. Two responses indicated that, client/customer can’t prioritise requirements

Spiral lifecycle

Spiral is the fifth investigated method in the report.

The tables and figures below, shows the results of the investigated projects that used Spiral

lifecycle. Table 36 shows the number of participants that used spiral, techniques, project type,

project size and organisation size. The results show that, projects used spiral are small to medium

sizes, with small and medium organisation sizes.

The result shows that, the techniques used by projects are four: prototyping, OO, Coding

techniques and Structure analysis/design. All the techniques used with spiral are the techniques

used in the questionnaire.

Clearly the result shows that there are some sorts of agreement on the use of the techniques with

spiral among participants.


- 143 - Last Saved 7/06/2010 11:24 PM Status: Draft



1 Coding techniques Intranet, Transaction processing, mainframe, Database

10 1

2 Prototyping, Coding techniques, Structure analysis/design



Management information system, Decision support

30 0 40 0

4 Prototyping Internet 1 1 1 0

Spiral

Projects

5 Prototyping, OO, Coding techniques, Structure analysis/design

Management information system, Decision support, Sensor data fusion

22 1 24 0

Table 36: Spiral & Techniques (by project and organisation sizes)



Investigated Spiral Projects The results of the 2 figures stated below, concerning the quality of the investigated projects, with

correlation to table 38 below, the results shows that, the three projects are completed, the

duration of the projects are 4 months, 4 months and 18 months respectively. Concerning the

quality of the produced product (system/application), 3 questions (refer to questions 16, 32 and

33) in the questionnaire have been set to collect data from participants, in order to measure and

determine product quality, the completion of the product (e.g. project signoff, project in

production), by asking participants about the product quality and by the number of defects in the

product, respectively.

These questions should give strong evidence if spiral will fulfil customer’s needs/requirements,

and that, customer will be satisfied with the final outcomes.


Spiral - Determine completion of projects

Project 3

Project 2

Project 1

System meets stated performance

Figure 94: Spiral – determine the completion of project

Figure 94 is about the methods used by participants in order to determine the completion of

projects, the colours refer to projects. The figure shows that, none of the participants have chosen

both methods, one project response with ‘other’, for this project, participant did not use a

standard method to determine the completion of the project, as the one used in the questionnaire.



And one participant (project 1) responded with “Project in testing”, this is not the first time we

have seen that participants used “Project in testing” as a completion of the project.

Spiral - Determine completion of projects by defects

0

1

2

3

Less than 5 defects Between 5 and 20bugs



Other

Project 3

Project 2

Project 1

Figure 95: Spiral – determine the completion of projects by defects

Figure 95 shows that, the same as the other lifecycles, none of the projects are free of defects.

Project 1 responded with less than 5 defects, in correlation to result in figure 94, we can assume,

why participant determine the completion of the project in testing. This is one of the weakness in

the questionnaire, I did not indicate the type of testing, such as User acceptance testing etc. this

point should be included in future study, again we correlate this result with figure 99, which

reflect the analysis of question 32 (concerning product’s quality), the indicated quality for project

1 point to high quality.

The other 2 projects 2 and 3 indicated the level of defects between 5 and 20, and again from

figure 99 the indicated quality for both projects are, high and very high quality respectively.

No evidence found to support these criterions.



Project complexity

Concerning complexity in the investigated projects, 2 questions (question 23 and 24 in the

questionnaire) have been set, asking participants to provide data about their projects.

Table 37 below show the results collected from question 23. The question set to collect

information from participants about the complexity in the projects.

As I’ve mentioned in the pervious analysis, two common ways used to determine the complexity.

The data shown in table 37 (first criteria) is exactly as given by participants. Figure 96 below

(second criteria) illustrates the results from question 24. It shows the level of complexity of each

project.

Management tools Used



Testing tools Used

Programming language Used

Design tools

Open source tools

Source control tools Used

Project 1

Other

Management tools



Testing tools

Programming language Perl,html,css,JS,mysql

Design tools

Open source tools Quanta, Perl,, MySQL, apache

Source control tools Quanta

Project 2

Other



Management tools MS Project and Office

Automation testing tools In house

Case Modelling tools Matlab and in house tools

Testing tools In house

Programming language Assembler, C and C++

Design tools Documentation tools, Smartdraw and MS Office.

Open source tools GNU editor and tools

Source control tools CVS, TKCVS and SourceSafe

Project 3

Other User Interface Development Tools Table 37: Spiral - Project complexity first criteria

Table 37 above shows the variety of complexity used within the projects, lets have a look to

programming languages used in the projects, this is also will help to find out the suitable

programming with spiral.

Participant of project 1 did not response with any data.

The programming languages shown in the table for the other two projects are: (Perl, html, CSS,

JS, MySQL),( Assembler, C and C++).

The programming languages used in the projects, are different from the languages used in other

projects. Also, the languages vary from old to new programming languages.


The colour in figure 96 reflects/indicates the response (levels of complexity) in projects,

collected from question 24, attached to the graph (question 24 in the questionnaire).


statement selected in answering the question, which is corresponding the marked statements in

the question.

The results are very clear except for project 2 responded with ‘other’ and show the complexity in

the project by “LAMP application”.

I couldn’t find any evidence that support these criteria.



A F A

B

1 2 3Projects

Spiral - Complexity of Projects

F

E

D

C

B

A

ABCDEF

Figure 96: Spiral – Project complexity second criteria

Rate of change in the projects and constraints Concerning the status of the received requirements at the beginning of the projects, and the rate

for requests of changes received are the criteria used to define the rate of changes in projects.

Five questions (25, 26, 27, 28 and 29) set in the questionnaire to collect data from participants.

The results of these questions have been presented in figures 97 and 98.

Figure 97 illustrate the results of the collected responses about the requirements status at the

beginning of the projects (question 25 – questionnaire). The coloured bars reflect requirements

status (as in the legend) at the beginning of the project. The results of this question with

correlation with the questions stated above, all together should help to specify the rate of changes

in the projects.

The result shows that, project 3 have received complete requirements at the beginning of the

project, projects 1 and 2 received partial and ambiguous requirements. Again connecting this

result with result illustrated at figure 99, we find that, the project received complete requirements

has produced product with very high quality, and the other 2 projects produced products with

high quality.




1 2 3Projects

Incremental - Project requirements defined at the beginning of the project

Complete

Ambiguous

Partial

Incomplete

Figure 97: Spiral – Projects requirements defined at the beginning of the project

1 2 3

2 0

1 10

5

17

5

3 3

15

Projects


Question 26

Question 27

Question 28

Question 29

Figure 98: Spiral – Rate of change in projects by request

Preceding with the analysis, table 12 (Appendix J), shows the questions (26-29) that addressing

the rate of changes in the projects, (refer to questionnaire). The result of these questions is



presented at figure 98. The figure shows 4 rows and 3 columns, the column number refer to the

participated projects, every row has a unique colour which reflects one question of the 4

questions mentioned above.

The numbers on the top of the bars indicates the number of requests for changes received for

each project, on each question.


Spiral - Levelof quaity expected in the project

1

1

1

1

23

Very highquality

High quality

Good quality

Low quality

No quality

Figure 99: Spiral – Level of quality expected in the projects

Figure 99 show projects quality, the colour and the number at the top of the axes refer to the

project number. At the centre of the chart the quality starts at zero (no quality) and then it

increase till reach to the highest marks in axes, where the indicators reach to the value of very

high quality (such as for project 3).

The figure shows that, the qualities expected of the three projects are high, high and very high.






1 YES 4 months

2 YES 4 months

3 YES 18 months Table 39: Spiral - Investigated project status and duration

Table 39 shows the status of the investigated spiral projects and its durations, by project number.

Project criticality

Table 40 below shows how critical is the failure of the project to the organisation. The result

show, project 3 is very critical and the failure could cause potential loss of human life. Clearly

we can see that spiral lifecycle is suitable for high critical projects.





Nothing much Other

1 Yes 2 Yes 3 Yes Yes

Table 40: Spiral – How critical is the failure of project to the organisation

Spiral - Projects budget

0 1 2 3 4

Generous budget

Sufficient budget

Insufficient budget

No budget

Don't know

Number of Projects

Projects 1

Projects 2

Projects 3



Figure 100: Spiral – Projects budget

Figure 100 represent the result of project’s budget. The data has been collected from the

responses of question 30.

The graph shows that, all the investigated projects have sufficient budget.

Project budget has an influence on the development lifecycles and on the project quality.

I have found no evidence that support these criteria.


Spiral - Team experience

0 1 2 3


High experience

Good experience

Little experience


Number of projects

Project 1

Project 2

Project 3

Figure 101: Spiral – Project team’s experience

Figure 101 shows the result for the level of experience of the project’s team. It shows the level of

experience for each team as well as the number of projects at each level.

Question 35 has been set to collect responses from participants.

It is obvious that, the more experienced is the team, the more successful of using the software

development methodologies, until the opposite proven.

The result shows clearly that, all the teams are with high experience.

The same as before I couldn’t found evidence to support these criteria.




participants to measure the level of team’s involvement in decision making within the project.

Spiral - Team empowerment on the projects

0 1 2 3 4

Always

Often

Rarely

Sometimes

Never

Number of projects

Project 1

Project 2

Project 3

Figure 102: Spiral – Team empowerment

Figure 102 below shows the result of the collected data from question 36 in the questionnaire.

The graph illustrates the number of projects and the team involvement level. The colours in the

legend are to connect projects to their corresponding level of involvement (e.g. project 1 etc).

From the analysed results, all teams have a strong involvement in decision making.

The question raise again, does spiral support team empowerment?


Figure 103 shows the result of team meetings fortnightly, e.g. for project 1 the team conduct 1

meeting a fortnightly, projects 2 and 3 conducting 2 meetings fortnightly.

The frequent of the team meetings for all of the projects fit very well with the analysis mentioned

above for these projects.

I’ve found no evidence to support these criteria



0 1 2 3 4


1

2

3

Proj

ects

Spiral - Team meetings (fortnightly)

Figure 103: Spiral – Frequent team meeting (fortnightly)

Spiral - Common communication methods among team members

0 1 2


Mostly formal


Mostly Informal


Number of projects

Project 1

Project 2

Project 3

Figure 104: Spiral – Common communication methods among team members



Figure 104 shows the results about communication methods used between team members.

Question 38 in the questionnaire has been set to achieve this purpose. The graph shows the

communication methods used by all projects. Team communication, meetings and involvement

in decision making helps the development team to achieve effective delivery of product with

good quality.

The result shows that, project 3 using mixture of formal communication method, project 1 using

mostly formal and project 2 using formal communication methods.

No evidence found to support the criteria.

Spiral - Common communication methods between team members and managers

0 1 2 3


Mostly formal


Mostly Informal


Number of projects

Project 1

Project 2

Project 3

Figure 105: Spiral – Common communication methods between team members and managers

Continue from the analysis above, figure 105 shows the result of the investigated projects for the

communication methods between team members and managers (refer to question 39 in the

questionnaire). Projects 2 and 3 used mixture of formal and informal communication method.

Project 1 used the same communication method between both team members and manager.




Spiral - Project management/Leadership main roles on projects

0 1 2 3 4

1

2

3

Expe

rinec

e &

Pro

ject

No

Number of roles

Programmer

Designer Analyst

Tester

Team Leader

Business Analyst

Project Manager

Architect

Other

First time

3 years

7 years

Figure 106: Spiral – Project Management/Leadership main roles and experience

Figure 106 shows the results of the leadership roles on projects. Questions 40 and 41 in the

questionnaire are designed to collect data from participants of their roles, the roles expected such

as project managers, team leaders, programmers, analysts, designers, architects etc.

Project 1 indicates that the participant’s roles on this project are designer analyst, project 2 and 3

have three roles on the project.



development lifecycles. All of the three roles are important to the project.

The graph shows very straight forward results, for the software development lifecycle (SDLC)

experience.

E.g. project 3 shows that participant has 12 years of experience in software development,

business and SDLC.




Spiral - Project Management/Leadership experience - Project Management/Leadership experience

0 1 2 3 4 5 6 7 8 9 10 11 12 13

1

2

3

Proj

ects

Number of years



SDLC experience

Figure 107: Spiral – Project Management/Leadership experience

0 1 2 3 4 5 6

Techniques

1

2

3

Proj

ects

Spiral - Techniques used in projects

Prototyping

OO

Coding techniques


Cleanroom

Pair Programming

Other

Figure 108: Spiral – Techniques used in projects




questionnaire are the common techniques used in industry.

E.g. the result shows that, the three projects have used prototyping technique.

No evidence found to support this criterion for iterative.

Now, concerning the advantages and disadvantages of spiral lifecycle used in the investigated

projects, two questions has been set to collect data from participants, questions 48 and 49, the

results are shown in figures 109 and 110 respectively.

Spiral - Advantages/Benefits of the used lifecycles

0 1 2 3 4 5 6 7 8

1

2

3

Proj

ects

Advantages

A

B

C

D

E

F

G

H

I

J

K

Figure 109: Spiral – Advantages/benefits of the used lifecycles

Figure 109 illustrates the advantages of spiral lifecycle used in the investigated projects. The

graph is very straightforward.

Description of the figures, the items A, B, C, D etc, represent the statements in the questions 48

and 49 in the questionnaire. The letter K represents respond ‘other’



Spiral - Disadvantages/Limitations of the used lifecycles

0 1 2 3

1

2

3

Proj

ects

Disadvantages

A

B

C

D

E

F

G

H

I

J

K

Figure 110: Spiral – Disadvantages/limitations of the used lifecycles

Last question in the questionnaire is question 50, I have asked participants “under which

condition particular development methodologies should not be used”. The responses collected

from this question are illustrated in the table 41 (Appendix H).

Findings – Spiral lifecycle

My founding will be based on the software development lifecycle under investigation. And on

the light of the analysis carried out on spiral lifecycle, the list of founds are:

1. I’ve found that the spiral lifecycle have been used in small, medium and large projects, as

well as in small and medium organisations (see table 36).

2. Spiral lifecycle has been used for different types of projects, the project types illustrated in

the questionnaire (see table 36). This finding should be investigated further in future study.

3. Spiral lifecycle is suitable for high critical projects

4. All the participated projects have used the techniques illustrated in the questionnaire.

5. All participants used spiral have high or above experience.



6. From table 37, I have found that, projects have used both new and old programming

languages.

7. I have found no evidence to support the assumption that project factors such as size, time

pressure, complexity, quality, budget and technology affects the use of spiral lifecycle.

8. The results founded from advantages and disadvantages of spiral lifecycle illustrated at

figures 99 & 100.

9. Assumption base on the analysed results, spiral can produce product with good quality, if the

spiral projects established on the base of high experience teams, leadership has experience in

the methodology, received stable requirements, and changes in requirements under control,

project complexity is not high.

Conclusions

The survey was conducted for the software industry in both the public sector and private sector.

There were, 29 responses, 10 responses from public sector and 19 responses from private. The

responses show that the development teams in private sector are more experience than the

development teams in public sector.

All the results from the analysis are listed under each investigated lifecycle above.

One participant’s response did not select any development methodology, because the participant

worked on testing project (member of testing team). Another participant sent comment by email

“It was quite hard to choose a lifecycle, as nothing formal is in place, so even though I'd like to

say "chaos", I think what I've chosen is more fair. I'd like to add a personal note though:”,

indicating no particular lifecycle was being in the company, but participant participated in the

survey based on previous experience in other places.

The results also show that four participants indicated they have no experience in development

lifecycles. Based on all of that I will conclude the following: first, some of the responses do not

provide true value to the project, second, participants who were not directly involved with

software development methodologies, also did not provide true value to the project.



I must address some issues: the survey was comprehensive and met project objective, but during

the analysis, I found that some aspects needed to be explored further, which will lead to an

expansion of the questionnaire for future research. For example, in respect to organisation

culture, the questions should target particular methodologies and not be general, in order to get

responses that address the usage of particular methodology with the work place in use. Also, in

order to identify any failure of the usage of particular methodology, questions should target this

issue directly, such as, if project management of the project under investigation has changed the

development methodology during project implementation and why.

For every investigated lifecycle I have listed findings, advantages and disadvantages of the

lifecycle in the analysis. Other findings, observed points will be stated in the conclusion

I needed to do a comparison with other survey results, but I couldn’t find similar surveys to

compare with, but to cover this point, I used a literature review that addressed software

development methodologies and the aspects surrounding them.

The use of organisational culture factors in the survey were intended to prove an assumption, that

is that culture effects/influences or determines the use of development methodologies. The

analysed data show that, no clear evidence was find to support the assumption, maybe because

the effect of organisation culture/environment on methodologies is not targeting one direction.

Also should consider the assumption of methodology effect on project environment (so

management could change methodology type).

Other factors were observed on the use of lifecycles and techniques by different organisations

sizes and teams’ sizes, the results shown that, methodology not used on the suitability of

particular organisation size or team size. Therefore, it was difficult to draw conclusion on the

relationship of these factors on development methodologies.

Also no evidence found to support the assumption of the influence/effects of project factors such

as complexity, quality, time pressure, project size, budget, project type, technology, change in

requirements and project criticality on the use of particular lifecycle.

The results show that, some organisations do not pay much attention to software development

methodologies in our software industry (warning sign).



The results also show that, one of the reasons behind changing the chosen methodology in the

project is, when the customer cannot understand the needs of the development methodology.

Also the results show that, waterfall are not useful method to be used in project were project

requirements/goals are not mature.

Problem in implementing incremental, I will draw strong conclusion based on the given data on

the investigated project (project 2) of incremental lifecycle. The team worked on project 2 fails

to enforce any process during the implementation of incremental. The question raise here, what

are the factors that cause the failure? If we consider the given data of the project we note the

following:

1. The Leadership have only 2 years of experience in incremental

2. Leadership role on project is programmer

3. Informal communication method between management and team

4. Mostly informal communication method between team members

5. Team meet once a fortnight

6. The team have little experience

7. No budget assign to this project

8. No deadlines set for this project

9. Incomplete requirements received at the beginning of the project

10. Some team members are unfamiliar with the development methodology and with the

development tools

Therefore, I can conclude that, incremental required high experienced leadership and high

experienced team. Also the team have used informal communication method. [5] Stated “

The most efficient and effective method of conveying information to and within a development

team is face-to-face conversation” this is an agile principle, but it is not limited to agile, it look

like this method in practice did not help enforcing process during the implementation, or the

implementation of incremental is more likely to work with mixture of communication method.



Also team meet once a fortnight (especially in such situation) is not enough, frequent team

meeting is more preferable and more practical e.g. regular team meeting bring team members,

managers and customers/stakeholders together to think about the implementation process.

The entire factors listed above have an influence on incremental lifecycle.

Results show slight evidence of using combination of lifecycles in one project. Three projects

have used two lifecycles. First project have used waterfall and incremental. Second project have

used Incremental and agile. Third project have used waterfall and iterative. For the use of

techniques and the projects size, refer to figure 74: Incremental - techniques used in projects

(projects 4 and 5), and figure 91: Iterative – techniques used in projects respectively.

Finally, I should say that, project objective is achieved and the results of the survey were

adequate.

Future research & study

Due to the limitation of the project objectives, and the limitation in the project time, further

study, investigation and research need to be done on software development methodologies in our

industry.

I can say that this study is a step towards improving future research objectives and analysis of

data, in order to improve software industry productivity by paying more attention to the use of

software development methodologies.

This research project explores some hidden issues in our software industry, such as the lack of

experience, organisation/management/leadership styles and the neglecting of software

development methodologies. The knowledge and experience of practicing/implementing

software development methodologies should be strength and pass from generation to generation.

My finding is not the end of the journey, it is good step for exploring many things further, also

very useful to be used as base for further study.

The following issues should be explored further in future research:



1. The relationship and effect of organisation size (small, medium and large) on development methodologies in more detail

2. The relationship and effect of project size (small, medium and large) on development methodologies in more detail

3. The relationship between project type and development methodology

4. Explore the relationship between techniques and lifecycles in more detail

5. Every agile methods (e.g. XP, Scrum), has its own characteristics and purpose of use. Future study should look closely at this issue.

6. Agile methods need researching separately

7. The effect of rate of change in the project on methodology

8. The suitability of programming languages with lifecycles

9. The use of combination methods in one project



Appendix A

Software Development Methodologies user table Software development methodologies user table Agile

Waterfall Iterative Spiral Incremental

,

, Techniques Prototyping

OO

CT

SA&D

Clean room

PP

Prototyping

OO

CT

SA&D

Clean room

PP

Prototyping

OO

CT

SA&D

Clean room

PP

Prototyping

OO

CT

SA&D

Clean room

PP

Prototyping

OO

CT

SA&D

Clean room

PP

Prototyping

OO

CT

SA&D

Clean room

PP

Project size Small project

Medium project

Large project

Small project

Medium project

Large project

Small project

Medium project

Large project

Small project

Medium project

Large project

Small project

Medium project

Large project

Small project

Medium project

Large project Number of developers 1 – 5

6 – 11

12 or more

1 – 5

6 – 11

12 or more

1 – 5

6 – 11

12 or more

1 – 5

6 – 11

12 or more

1 – 5

6 – 11

12 or more

1 – 5

6 – 11

12 or more Project 1 – 6 months 1 – 6 months 1 – 6 months 1 – 6 months 1 – 6 months 1 – 6 months



duration 6 mnths – 1 year

1 – 2 year

2 – 5 years

over 5 years

6 mnths – 1 year

1 – 2 years

2 – 5 years

over 5 years

6 mnths – 1 year

1 – 2 years

2 – 5 years

over 5 years

6 mnths – 1 year

1 – 2 years

2 – 5 years

over 5 years

6 mnths – 1 year

1 – 2 years

2 – 5 years

over 5 years

6 mnths – 1 year

1 – 2 years

2 – 5 years

over 5 years Length of iteration

OO=Object-Oriented, CT=Coding Techniques, SA&D= Structure Analysis & Design, PP=Pair Programming

In additional to the table above and in order to use software development methodologies, user must refer to the analysis at section 4, particularly look to strength (advantages), weakness (disadvantages), when to use and not to use any of the lifecycles.


167

Appendix B

Questionnaire


Overview The Australian software development industry is diverse. Small to medium enterprises (SMEs) engage in software projects ranging from small, simple database applications to medium, complex internet games, and on up to very complex embedded software for (tele)communications systems. Some software developments in Australia are world-beaters, especially games and financial systems. In the School of Computer Science (SoCS) at the Australian National University (ANU), educators and researchers alike are attempting to discover and to teach new, practical ways to develop software. Both undergraduate and postgraduate courses are run in computer science and software engineering. As such it is important to be able to inform students of current practices and trends, and also to research and evaluate the effects of changing practices and technology.

Purpose This questionnaire is part of a masters student research project being conducted within SoCS at ANU in order to discover the types of software development methodologies currently being used for new software developments, especially as used/adopted by SMEs. Methodology effectiveness, frequency of use, and flexibility of methods, are just some of the important properties to be studied after obtaining responses to this questionnaire. Responses can be provided in anonymity, but in any case no particular results associated with any single company, or set of companies, will be divulged in the final report or in other communication with any organisation or individual. The research is focussed on identifying overall Australian software development industry trends. By taking part in this study, you will provide valuable information for, not only the software industry sector but also other industry sectors reliant on software. The information will also be valuable for our future software developers/engineers who are undertaking information technology, software engineering and other related degrees at various universities in Australia.

Who should respond This questionnaire is intended for professional software developers (in organisations) who typically work within a software team, such as a team leader, business analyst, software architect, designer or project manager. Responses to the whole questionnaire should take no more than 20-30 minutes to provide. More than one person can provide answers when no single individual is able to answer all questions. If any question cannot be answered, for any reason, then simply move on to the next question. If there are any concerns, issues, or questions surrounding this questionnaire then please either email [email protected] or call Dr. Clive Boughton on 0410 632 055. Dr. Boughton is the supervisor of the student undertaking the research for his masters.


168

Thank you for participating in this study. Section 1: Organisational Factors This section of the questionnaire is intended to gather information about the development section in your organisation, company or department. Please note that all yellow coloured fields are textually editable. Simply (mouse) click on the boxes/fields that you would like to tick. Once you have completed clicking/ticking boxes or typing text in the yellow fields, please save the file before returning to Ahmad Deeb [email protected], or send your hard copy to P.O Box 6304, CONDER, ACT 2906. Please return the questionnaire as soon as possible, or no later than Sunday 20/04/2010 1. What type of development does your organisation/section typically undertake?

Please tick all that apply We provide specific software products/services to customers by:

Producing complete software/applications

Customising pre-developed software

Providing service/support for our own software systems/products

Providing service/support for other software systems/products

Other, please specify:

We provide in-house development to support running of the organisation by:

Carrying out our own development

Purchasing and customising externally developed products


We develop software to meet government policies/legislation to serve the public by:

Doing in-house development

Purchasing software from an outside source (software vender)

Developing our software outside the organisation (with private company)


2. Approximately how many software developers (permanents/contractors) are there within

your organisation?

Permanent = Contractors = OR,

If you don’t know please tick this box Lifecycles

3. Software development lifecycles are typically chosen depending on the type of project being

run. Additionally, the state of the requirements may dictate the type of lifecycle that needs to


169

be applied initially as opposed to changing to a different lifecycle once requirements stabilise.

In the following table, the most common development lifecycles are listed (on the left). If your organisation uses any of these lifecycles please indicate on how many of the last 8 projects they’ve been used by ticking one or more boxes.

As an example; if your organisation used the waterfall lifecycle on 3 of the last 8 projects, and the incremental lifecycle on another 4, it would be appropriate to tick the box that corresponds with the intersection of the row labelled “Waterfall” and the column labelled “3 projects” and then tick the box that corresponds with the intersection of the row labelled “Incremental” and the column labelled “4 projects”. Of course there is also the possibility that a project might have started using an “Agile’ lifecycle but then changed to “Waterfall” once requirements were fixed. In that case tick two boxes in (say) the “1 project” column.

Please tick all that apply.

Lifecycle 1 project

2 projects

3 projects

4 projects

5 projects

6 projects

7 projects

8 projects N/A

Waterfall

Spiral

Incremental

Iterative

Agile

Other

4. If you indicated having used an “Agile” lifecycle in the table above, then please name 1-3 particular agile approach(es) your organisation applied. Explanation: Agile has different development approaches/methods and techniques. e.g. Extreme Programming (XP), SCRUM etc. (for more details refer to appendixes A & B.

Agile approach(es) used:

5. If you indicated having used an “Other” lifecycle in the table above, then please name 1-3 particular approach(es) your organisation applied (e.g., In-house standard, Method One, etc.)

Other approach(es) used:


170

Methods (Techniques)

There are many methods available to be used at different stages in a software development lifecycle. Some have been around for two or more decades whilst others have appeared more recently. Although there are many actual products available, most support one of a few actual methods. Often the lifecycle that is used for a development suggests the sort of method that should be used and vice-versa. However some organisations maintain particular skills in just one method and often apply the same lifecycle for each project.

The table below shows 5 typical lifecycles as row headings, and lists 6 typical methods that are typically used in different parts (or sometimes the whole) of the software development lifecycle.

6. For each software development lifecycle that is used within your organisation, please tick one or more boxes corresponding to the method(s) that are typically applied.

Lifecycle Waterfall Spiral Incremental Iterative Agile

Techniques

Prototyping

Object-oriented

Coding techniques

Structure analysis /design

Clean room

Pair programming

Other, pls name

Prototyping

Object-oriented

Coding techniques


Clean room

Pair programming

Other, pls name

Prototyping

Object-oriented

Coding techniques


Clean room

Pair programming

Other, pls name

Prototyping

Object-oriented

Coding techniques


Clean room

Pair programming

Other, pls name

Prototyping

Object-oriented

Coding techniques


Clean room

Pair programming

Other, pls name


171

Organisational Culture The frequent (perhaps unthinking) use of particular development methodologies and lifecycles can lead to an organisational culture that stifles opportunities for effective change. Attempts to adopt or experiment with new, perhaps more effective methods and lifecycle combinations nearly all tend to ‘fail’. The software development part of the organisation can become structurally inert and be lead by people who resist trying out new ideas – after all they know what’s right! An open organisational, informal and non-hierarchical culture indicates a high degree of organisational maturity. However, there are lots of factors that affect culture. The following tables presents statements about organisations, leadership, management, attitudes, people focus and more. Please think of each statement as it applies to your organisation, and then tick one of the 5 boxes indicating your level of agreement with the statement.

7. Please indicate your level of agreement with respect to the nature or characterisation of your organisation.

StronglyAgree Agree Neutral Disagree Strongly

Disagree

The work environment is very comfortable and a nice place to be. The organisation treats staff as an extended family and supports social activities.

The organisation is a dynamic and entrepreneurial place.

The organisation is results oriented. The major concern is with getting the job done. Staff are very competitive and achievement oriented.

The organisation is well controlled and structured. Formal procedures govern what staff do.


172

8. Please indicate your level of agreement with respect to your organisation’s leadership.


Disagree

The leadership is mentoring, facilitating, nurturing and promoting.

The leadership is innovative and willing to take risks in order to make a profit.

The leadership is practical, challenging and results oriented.

The leadership is coordinated, organised and smooth running.

9. Please indicate your level of agreement with respect to your organisation’s management

qualities.


Disagree

Staff management is characterised by teamwork, work agreement and participation.

Staff management is characterised by individual risk taking, innovation, freedom and individualism.

Staff management is characterised by hard driving competitiveness, high demands and achievement.

Organisation of staff is characterised by security of employment, conformity, predictability and stability in relationships.


173

10. Please indicate your level of agreement with respect to your organisation’s management

style/qualities. The keys that hold the organisation together are:


Disagree

Loyalty and mutual trust. Commitment to the organisation is very high.

Commitment to innovation and development. There is an emphasis/desire for being on the cutting edge.

Emphasis on achievement and goals.

Formal rules and policies. Maintaining a smooth running organisation is most important.

11. Please indicate your level of agreement with respect to your organisation’s emphasis.


Disagree

Human development, high respect, high trust, openness and participation persist.

Acquiring new resources and creating new challenges. Trying new things and prospecting for opportunities is valued.

Competitive actions and achievement.

Performance and stability. Efficiency, control and smooth operations are most important.


174

12. Please indicate your level of agreement with respect to your organisation’s definition of success:


Disagree

Development of excellent/competent human resources, who are committed, and teamwork oriented and who provide excellent services.

Producing high quality products to serve its customers, who in turn are very satisfied.

Innovation and uniqueness in its field, producing the newest product, and being “ahead of the pack.”

Winning national awards of excellence, being very competitive in its field.

Section 2: Project description This section is to gather information about projects. Please select one typical project which was recently completed or is currently underway within your organisation and is using a software development lifecycle (and methodology).

Naming a project is optional (The intention is only to draw a participant’s attention and concentration toward/on a particular project)

13. What is the project name?

14. Please tick the project status and indicate the duration of the project.

Project completed. Duration of the project:

Project not completed. Estimated duration:

15. What is the project? Please select one of the following options.

New system

Upgrade of an existing system

Maintaining a system

Replacement of an existing system

Other, please specify


175

16. What criteria are to be or have been used to determine the completion of the project? Please select one of the following options:

Project is in use in Production environment

System is in testing environment or under testing

Customer signoff


17. How many full time staff are involved in the project (include developers, testers, analysts, managers etc)?

18. How many part time staff are involve in the project (include developers, testers, analysts,

managers etc)? 19. Approximately how many end-users will use the system/application developed in this project?

OR, please tick this box if you don’t know

20. Please describe briefly the purpose of the project. (For example: The system will be used by Australian citizens to enable them to lodge passport applications over the internet).

21. Which sector does the project serve or address?

Public service Defence

Telecommunications Engineering

Sales Marketing

Gaming Banking/finance

Data warehousing Other, please specify

22. What type of application/system will the project produce?

Internet application/system

Intranet application/system

Transaction processing

Management information system

Decision support system

Operating system

Mainframe application/system


176

Database

Data mining or business intelligence


Project complexity

23. What specific tools have been used in the project?

Project management tools

Automated testing tools

Case/Modelling tools

Testing tools

Programming language(s)

Design tools

Open source tools


Other tools

24. Indicate the level of complexity of this project? Please, tick all that apply.

Integration of more than two applications

More than two new technologies involved that the development team has not used previously

Some team members are unfamiliar with the business roles or project domain

Some of the developers unfamiliar with the development tools

Some team members are unfamiliar with the development methodology


Rate of change in the project 25. How well were the project requirements defined at the beginning of the project (include user and

business requirements)?

Incomplete requirements were provided

Partial/segmented requirements were provided

Ambiguous requirements were provided

Complete requirements were provided



177

26. How many requests have been received for adding new requirements or additional functionality since the project started? (For example: changing a feature to include a new calculation, or a new feature to transfer data to an external server, or a new feature which allows a user to administer the system etc.)

27. How many requests for corrections/changes to existing functional requirements have been received since the project started? (For example: a change to the interface of the application, or a change to calculation tax rate for the application etc)

28. How many requests for changes to existing technology requirements have been received since

the project started? (For example: changing the development language, change of database/vendor etc)

29. How many requests for changes to other project factors have been received since the project

started? (For example: changes to the budget, delivery date, staff etc)

Project constraints 30. To what level is the budget for this project adequate to achieve its business requirements?

Generous (more than enough)

Sufficient

Insufficient

There are no defined budgeting constraints for this project

Don’t know

31. What time pressure do you think this project was or is currently under?

Extreme time pressure (impossible deadline was set)

High time pressure (a very tight deadline was set)

Moderate time pressure (a tight, but achievable deadline was set)

Low time pressure (a comfortable deadline was set)

No time pressure (a very flexible deadline was set)


178

32. What is the level of quality expected of the application developed?

Very high level of quality expected (e.g. Tax systems, Banks systems, Transports etc. - No

serious errors & very few trivial errors)

High level of quality expected (No serious errors & few trivial errors)

Good level of quality expected (possibly a relatively serious error & several trivial errors)

Low level of quality expected (one or more serious errors & several trivial errors)

Quality is not specified/considered for this project

33. Indicate the expected/measured level of defects in the application when deployed?

Less than 5 defects when deployed (with no system failure)

Between 5 and 20 bugs (with no system failure)




Project criticality 34. How critical is the failure of this project to the organisation, if a serious defect is found in the

delivered/deployed product?




Nothing much


Team factors in the project

35. Indicate the level of experience of the project team (include development team and business area experience)

Very high experience (most of the team has more than 10 years relevant experience)

High experience (most of the team has more than 5 years relevant experience)

Good experience (most of the team has at least 3 years relevant experience)

Little experience (most of the team has less than 2 years relevant experience)

Virtually no experience (most of the team has one year or less relevant experience)


179

36. How frequently are project team members involved in making decisions about problems within the project? (This question assesses the extent of team empowerment on the project)

Always Rarely

Often Never

Sometimes

37. How frequently has, or do the whole team or its representatives met, or meet in discussion with the client/customer(s)?

38. What is the most common communication method that has been, or is being used among team

members within the project?

Formal, mostly by formal document, formal meeting or email

Mostly formal, but with some informal interaction

Mixture of formal and informal communication

Mostly informal, but with some formal interaction

Informal, most communication is face to face

39. What is the most common communication method that has been, or is being used between team

members and project managers?

Formal, most communication by formal document, formal meeting or email

Mostly formal, but with some informal interaction

Mixture of formal and informal communication

Mostly informal, but with some formal interaction

Informal, most communication is face to face

Project management/leadership experience 40. What are the main role(s) that you have in this project?

Programmer Designer Analyst Tester Team Leader

Business Analyst Project Manager Architect



180

41. How long have you been performing the main role(s) you indicated in answer to Q40 (above)?

First time (on this project)

3 to 6 months

More than 6 months, up to 1 year

Up to years (state how many years)

42. How long have you been using the chosen development lifecycle/technique(s)? (e.g. Waterfall, Agile methods, Spiral, prototyping, structure analysis and design etc)


3 to 6 months



43. How much experience do you have within the business domain of this project?


3 to 6 months



44. How long have you been working in the software development field?


3 to 6 months




181

Usage of software development lifecycle (methodologies) within the project 45. Which development lifecycle(s) and technique(s) have you used for this project?

Development lifecycle Waterfall Spiral Incremental

Agile

(e.g. XP, scrum) Iterative

Techniques

Prototyping

Object-oriented

Coding techniques


Cleanroom

Pair programming

Other, pls specify

Prototyping

Object-oriented

Coding techniques


Cleanroom

Pair programming

Other, pls specify

Prototyping

Object-oriented

Coding techniques


Cleanroom

Pair programming

Other, pls specify

Prototyping

Object-oriented

Coding techniques


Cleanroom

Pair programming

Other, pls specify

Prototyping

Object-oriented

Coding techniques


Cleanroom

Pair programming

Other, pls specify

Used our organisation’s own development lifecycle (methodology), please go to question 46

46. If you indicated that your organisation’s lifecycle (methodology) was used, in answer to Q45 (above) then please briefly describe the

methodology/technique(s).

47. If Agile methods have been or are being used for this project, what is the typical length of iteration used?

1 week 2 weeks 3 weeks 4 weeks Other, please specify


182

Section 3: General Information Benefits of software development lifecycle (methodology) 48. From your experience, what are the benefits of the methodology used for this project?

(Please select one or more)

More control of the development process/schedule

Increased productivity (more productive team)

Reduction of effort

More customer satisfaction

Reduce resources (cost, time etc)

Delivery of good quality product

Improved communication between team members

Improved communication with customer/client

Shorter development time frame

Less bugs/defects in the product (application/system)


49. From your experience, what are the disadvantages/limitations of the methodology used for this project? (Please select one or more)

Requirements changes are difficult to control

Difficult to meet client/customer requirements/needs

Client/customer can’t prioritise requirements

Getting the support of management for the methodology technique is difficult

Poor design documentation

Some techniques are not acceptable to the team members

Controlling iterations is difficult

Delays with delivery of the product

Hard to control budget

Reduced team productivity



183

50. From your experience, under what conditions do you think particular development methodologies should not to be used?

Thank you very much for taking the time to provide answers to the range of questions in this survey. After analysis of all the answers from all parties invited to participate, you will be informed of the results. Ahmad Deeb, Clive Boughton


184

Appendix C

Questionnaire design Section 1: Lifecycles, Techniques and Organisational culture

This section will collect data form participants about the usage of lifecycles, techniques and organisation culture/environment (the physical environment of the work place)

Q1: This question using different factors to collect data from participants of the most common types of software development

Q2: This question to collect data from participants about organisation size, this question taken from the assumption “large development team in large organisations with formal working practices are more likely to use formalised methodologies” [14].

Q3 – 6: These questions are to collect data from participants about software development lifecycles and techniques. The questions will give indication on the most use methodologies in our industry.

Q7 – 12: These questions are to collect data from participants about organisations culture maybe is not suitable to be used. [16] Stated that, “agile methods should not be adopted if the culture is not suitable, otherwise the culture may need to be changed for the method to be successfully. The culture should be open, informal and non-hierarchical”. These questions adapted questions developed by [19] & [16].

Section 2: Investigation of individual Projects This section will investigate individual projects. This is an important section, which will

help to collect data from participants on using of lifecycles and techniques used in particular project.

Q13 – 22: Project’s name is to make participant focus on a particular project, Project duration is to help in the analysis by indicating if the project completed or not. The assumption I have, if the project is completed then it is more likely to get more accurate data from participants, “the use of the methodologies could be effectively assessed” [17]. The rest of the questions use factors to collect data from participants about project case, project completion, number of staff working on the project and number of end users using the final product, sector type and project type.

Project complexity Q23 - 24: These questions dealing with complexity in projects. The assumption “complexity in

software development cannot be directly determined, it is based on factors such as the novelty of the technology, the novelty of the business problem, team skills and the combination of technologies” [18]. Rate of change in the project

Q25 – 29: These questions are to collect data from participants about the rate of changes in projects such as, the number of requests received on changes in requirements, and project status at the beginning of the project.


185

Project constraints Q30: This question is to collect data from participants about project budget, this is an important

factor of any project, and also this factor indirectly reflects the importance of the project to the organisation.

Q31: This question to collect data from participants about the time pressure on the project. Q32 – 33: These questions are to collect data from participants about project quality. Project

quality is an important factor in software development. These questions are to find if there is any effects/relationship between quality and the development methodologies.

Project criticality Q34: This question is to collect data from participants about project criticality. The assumption is

for high critical projects, if “agile methods are not considered suitable for highly critical projects” [19], also will examine other lifecycles.

Team factors in the project Q35: This question is to collect data from participants about the level of experience in the team. Q36: This question is to collect data from participants about team empowerment on the project. Q37 – 39: These questions are to collect data from participants about frequent team meeting and

the communication methods used between team members and between management and team.

Project’s manager/leader experience Q40 – Q44: These questions are to collect data from participants about management /leadership

experience and their roles. Usage of software development methodologies within the project Q45: This question is to collect data from participants about the development methodologies

used in the project under investigation. Q46: This question is to find out if any other methodologies are used. Q47: This question is to collect data from participants, if agile methods are used in the project, as

well as the length of iterations used. Section 3 General Information Q48–Q49: These questions are to collect data from participants of the advantages and

disadvantages of the development methodologies used in the project. 50: This question is to collect data from participants on their opinion about the software

development methodologies used, and when it should not be used.


186

Appendix D

Email template The following email template has been used as the standard template for asking anyone to participate in the survey.

Dear A software engineering masters student at ANU (Ahmad Deeb) whom I am supervising, has decided to undertake a survey of (at least) the local software development industry in order to find out something (more than hearsay) about the sorts of development methods and project life cycles now being applied, and also some of the attitudes/conditions that exist surrounding organisations and their staff. To help in this endeavour both Ahmad and I would appreciate your input by answering some/all of the questions in the attached survey. Any information Ahmad and I receive from the survey will remain separated from the individual supplying it, but aggregated data that has been analysed for broad trends or other collective properties will be put together for the purposes of a report that Ahmad must submit for examination. Within the report, and any other communication, no mention of individual companies or individuals themselves will be made. The report shall be made available only to those who participate in the survey. However, depending on the outcomes and potential impact on the industry, permission will be sought from all participants of the survey for wider publication of the report. The survey has been planned by Ahmad over a period of months. Others including ex-industry adjuncts to the university and retired academics, have helped and guided Ahmad to the creation of the attached document (containing the survey). If you decide to respond, please read all background and instructions closely. If there are any questions that you either cannot, or wish not to answer, then please don't let that stop you from sending Ahmad and I the responses to only the questions you've answered. We understand that some people may feel very sensitive about some of the questions, and so we do not expect all individuals to answer all questions. At this stage it is important that Ahmad be able to obtain as many responses as he can in order to establish any reasonable outcome of his survey. So, to that extent I would ask you to not only respond to the survey yourself, but also that you ask two or three others, with similar experience and capability, to do the same. Both Ahmad and I would very much appreciate anything you can do to help increase the number of software professionals also able to provide input. Part of the purpose behind undertaking this survey is based on the realisation that over the past several years there seems to have been much change and maturation within the software development industry in both Australia and overseas - and especially among small to medium sized companies. It seems that to maintain a 'finger on the pulse' of industry requires researchers and academics to gather information effectively and efficiently. Hence the humble questionnaire/survey tool seems to be the most appropriate because more organisations and


187

individuals within industry can be contacted, and the time imposition on usually very busy people is largely minimised. Ahmad and I earnestly hope that you can spare a half hour to respond to the attached survey, so that some greater understanding of at least one or two aspects of progress in the Australian software industry can be studied. PLEASE NOTE: There are two attached documents, one in MicroSoft Word form (.doc) and the other in Adobe (.pdf) form. Both documents contain exactly the same information and so you need only to use and return one of them. The difference is that the MS Word document is enabled to accept your answers to most questions using mouse clicks, and there are some questions that need textual input which is typed into expandable text regions. This document can be completed and returned electronically - via email. The Adobe '.pdf' document is intended either for those who do not use MS Word, or who would prefer to return a hardcopy response. Of course, the choice is yours in any case. Ahmad and I thank you in anticipation of your willingness to participate in this survey. Best regards. Dr Clive Boughton Retired Australian National University CANBERRA ACT 0200 Australia Mobile Phone: +61 (0)410 632055


188

Appendix E

Questionnaire Feedback sheet


Questionnaire Feedback

Date: / / Name: (Optional) ------------------------------------------------------------------------------------------------------- Organisation: (Optional) -------------------------------------------------------------------------------------------------------- Comments:

Thank you for your feedback


189

Appendix F

Figures (SDM background – section 2)

Classical Waterfall model

Figure 1. Waterfall model with single-level feedback paths

Establish Requirements

Design Creation

Program Implementation

System Test

Release to Customer


190

Figure 2. Waterfall model

Figure 3. Spiral model for Software Development


191

Figure 4. Spiral model- Adapted from B.W.Boehm


192

Figure 5. Incremental for Software Development


193

Figure 6. Extreme Programming (XP) lifecycle

Figure 7. Scrum process


194

Figure 8. DSDM

Figure 9. Adaptive Software Development

Figure 10: Crystal family of methodologies


195

Appendix G

Organisation culture graphs and related tables

Indicator Description

A The work environment is very comfortable and a nice place to be. The organisation treats staff as an extended family and supports social activities.

B The organisation is a dynamic and entrepreneurial place.

C The organisation is results oriented. The major concern is with getting the job done. Staff are very competitive and achievement oriented.

D The organisation is well controlled and structured. Formal procedures govern what staff do.

Table 2: Organisation’s nature – question 7

Question 7 - Organisation's Characterisation (Public sector)

0

1

2

3

4

5

6

7

A B C D

Agree

Neutral Disagree

Strongly agree

Strongly agree

Strongly agreeAgree Agree

Agree

Neutral

Neutral

Neutral

Disagree

Disagree

Disagree

Figure 12: Organisation Characterisation (Public sector)


196

Question 7 - Organisation's Characterisation (Private sector)

0

1

2

3

4

5

6

7

8

9

10

A B C D

Agree

Neutral Neutral Neutral

Neutral

Agree

Agree

Agree

Disagree

Disagree

Disagree

Disagree

Strongly agree

Strongly agree

Strongly agree

Strongly agree

Strongly disagree

Strongly disagree

Strongly disagree

Figure 13: Organisation Characterisation (Private sector)

Question 8 - Organisation's Leadership (Public sector)

0

1

2

3

4

5

A B C D

Agree

Agree

AgreeStrongly agree

Strongly agree

Strongly agree

Strongly agree

Neutral

Neutral

Neutral NeutralDisagree Disagree

Disagree

Disagree

Strongly disagreeAgree

Figure 14: Organisation’s Leadership (Public sector)


197


A The leadership is mentoring, facilitating, nurturing and promoting

B The leadership is innovative and willing to take risks in order to make a profit

C The leadership is practical, challenging and results oriented

D The leadership is coordinated, organised and smooth running Table 3: Organisation’s Leadership – question 8

Question 8 - Organisation's Leadership (Private sector)

0

1

2

3

4

5

6

7

8

9

10

A B C D

Strongly agree

Strongly agree

Agree Agree

Agree

AgreeNeutral

Neutral

Neutral

Neutral

Disagree

Disagree Disagree

Strongly disagree

Strongly disagree

Strongly disagree

Strongly disagree

Figure 15: Organisation’s Leadership (Private sector)


A Staff management is characterised by teamwork, work agreement and participation

B Staff management is characterised by individual risk taking, innovation, freedom and individualism

C Staff management is characterised by hard driving competitiveness, high demands and achievement

D Organisation of staff is characterised by security of employment, conformity, predictability and stability in relationships

Table 4: Organisation’s Management – question 9


198

Question 9 - Organisation's Management quality (Public sector)

0

1

2

3

4

5

6

7

A B C D

Strongly agree

Strongly agree

Strongly agree

Agree

Agree Agree

Agree

Neutral

Neutral

NeutralNeutral

Disagree Disagree

Disagree DisagreeStrongly disagree Strongly

disagree

Figure 16: Organisation’s Management quality (Public sector)

Question 9 - Organisation's Management quality (Private sector)

0

1

2

3

4

5

6

7

8

9

10

11

A B C D

Strongly agree

Strongly agree

Strongly agree

Agree

Agree

Agree

Agree

Neutral

Neutral

Neutral

Neutral

Disagree

Disagree

Disagree

Disagree

Strongly disagree

Strongly disagree

Strongly disagree

Figure 17: Organisation’s Management quality (Private sector)


199


A Loyalty and mutual trust. Commitment to the organisation is very high

B Commitment to innovation and development. There is an emphasis/desire for being on the cutting edge

C Emphasis on achievement and goals

D Formal rules and policies. Maintaining a smooth running organisation is most important

Table 5: Organisation’s Management style/qualities – question 10

Question 10 - Organisation's Management style/qualities (Public sector)

0

1

2

3

4

5

6

A B C D

Strongly agree

Strongly agree

Strongly agree

Strongly agree

Agree

Agree

Agree

AgreeNeutral

Neutral

Neutral Neutral

Disagree

Disagree DisagreeStrongly disagree

Figure 18: Organisation’s Management styles (Public sector)


200

Question 10 - Organisation's Management style/qualities (Private sector)

0

1

2

3

4

5

6

7

8

9

A B C D

Strongly agree

Strongly agree

Strongly agree

Strongly agree

Agree

Agree

Agree Agree

Neutral

Neutral

Neutral

Neutral

Disagree

Disagree

Disagree

Disagree

Strongly disagree

Strongly disagree

Strongly disagree

Strongly disagree

Figure 19: Organisation’s Management styles (Private sector)

Question 11 - Organisation's Emphasis (Public sector)

0

1

2

3

4

5

6

A B C D

Strongly agree

Strongly agree

Strongly agreeAgree Agree

Agree

Agree

Neutral

Neutral

Neutral NeutralDisagree

Disagree Disagree

DisagreeStrongly disagree

Strongly disagree

Figure 20: Organisation’s Emphasis (Public sector)


201

Question 11 - Organisation's Emphasis (Private sector)

0

1

2

3

4

5

6

7

8

9

10

A B C D

Strongly agree

Strongly agree

Strongly agree

Strongly agree

Agree Agree

Agree Agree

Neutral

Neutral

Neutral

Neutral

Disagree

Disagree

Disagree Disagree

Strongly disagree

Strongly disagree

Strongly disagree

Strongly disagree

Figure 21: Organisation’s Emphasis (Private sector)


A Human development, high respect, high trust, openness and participation persist

B Acquiring new resources and creating new challenges. Trying new things and prospecting for opportunities is valued

C Competitive actions and achievement

D Performance and stability. Efficiency, control and smooth operations are most important

Table 6: Organisation’s Emphasis – question 11


A Development of excellent/competent human resources, who are committed, and teamwork oriented and who provide excellent services

B Producing high quality products to serve its customers, who in turn are very satisfied

C Innovation and uniqueness in its field, producing the newest product, and being “ahead of the pack.”

D Winning national awards of excellence, being very competitive in its field Table 7: Organisation’s Definition of success – question 12


202

Question 12 - Organisation's Definition of success (Public sector)

0

1

2

3

4

5

6

7

A B C D

Strongly agree

Strongly agree

Strongly agree

Agree

Agree Agree

Agree

Neutral Neutral

Neutral

Disagree

DisagreeDisagree

DisagreeStrongly disagree

Strongly disagree

Strongly disagree

Figure 22: Organisation’s Definition of success (Public sector)

Question 12 - Organisation's Definition of success (Private sector)

0

1

2

3

4

5

6

7

8

9

10

A B C D

Strongly agree

Strongly agree

Strongly agree

Strongly agree

Agree

Agree

Agree

Agree

Disagree

Disagree

Disagree

Disagree

Neutral

Neutral Neutral

Neutral

Strongly disagree

Strongly disagree

Strongly disagree

Figure 23: Organisation’s Definition of success (Private sector)


203

Appendix H

Participants’ responses on question 50 in the questionnaire

Waterfall projects

Project

No

Participant’s responses (Exactly as provided)

1 Strict Waterfall takes up too much time in gathering requirements (which change all

the time anyway). Due to the big outlay in time and resources at the beginning of a

project development work cannot begin and time is eaten up. Development as-you-go

in a more iterative approach allows for easy identification of errors and issues earlier

which in-turn allows for a full team be productive in both development and analysis

activities. Iterative approaches also provide opportunities for innovation and

enhancement to a product. Overly agile approaches can tax customers' time and are

not always suited to projects (in the case of a Data Warehouse frequent

iterations/interactions with the client are not as necessary as a pure software project.

Strict waterfall should never be used - always coupled with iterative/cyclic

approaches. Agile approaches are useful and powerful but must be exercised correctly

and appropriately.

2 Waterfall projects work well in civil engineering projects, because they are repeatable.

If your project is a repeatable one (i.e you've done something similar not so long ago

then do waterfall). IT projects are rarely repeatable exercises, in most occasions, you

assemble a team, with various experience in a project domain and various levels

of capability. Hence it is very difficult to estimated the amount of time and money that

it's going to take to make it happen. Most problems are not technological, but

organisational. Rigid business model, hierarchical command and control structures,

lack of incentives to innovate and investigate more productive ways, all of these

negatively impact projects run in particular organisation. It's a waterfall organisation

running waterfall projects.


204

3 The development methodology used is directly determined by the stability of the

requirements and the amount of change. If requirements are not stable, ill-defined or

subject to change the Waterfall model is not ideal. Conversely, where requirements are

very stable agile methods will appear to customers to be a waste of time.

4 XP not good when there is limited availability to talk to the client, but iterative isn't

good when the client doesn't know what they want.

5 For any project that is not going well, anything but Waterfall should not be used. Most

reach for the “quick” methodologies, but Waterfall is the best for getting an off-the-

rails project back on track

6 When they're not appropriate :) Don't try a waterfall when the customer doesn't

understand exact what they need (which is, naturally, different to what they ask for

and what they want).

The project I cite benefited from Big Design Up Front (BDUF). It was done

collaboratively with the whole team. The advantage here was that the design work

required knowledge of existing systems, in which the team was weak (I had it; the rest

did not). The team had great generic skills and great experience, so we used the design

exercise to build a shared understanding of what needed to be done. Further, since it

was a backend system where the customer was ourselves - traditional "interview the

client" BA techniques wouldn't work. The design document doubled as the functional

requirements.

7 Pair programming

10 Iterative not ideal for managing costs, Waterfall not ideal for constantly changing

requirements.

Table 16: Waterfall – Question 50 responses


205

Agile projects

Project

No

Participants responses (Exactly as provided)

1 It depends on the environment. In the enterprise systems environment (by which I

mean the environment where systems are built around an existing hardware and

software infrastructure, and aimed largely at automating or improving an existing

business process) any predictive methodology (such as waterfall) tends to be brittle,

expensive and largely ineffective. If however the organisation demands upfront costing

it is hard to avoid a waterfall model.

Agile should also be used with caution. It tends to emphasise the programming team

and de-emphasise the Architecture of the system. This fails to recognise the role that

architecture has in change management.

2 We use the idea of developing the smallest possible functional system to get real user

feedback and to evolve the product from that system with continual user experience

feedback is one that we believe is the most reliable and cost effective for our type of

product which involves many end users in many roles.

3 Waterfall is seldom recommended by industry to clients, but sometimes the client

mandates it as it is easier for them to track project completion. In these instances,

industry will typically at least break down the project into multiple waterfall iterations

to avoid a "big bang" go live, which is always considered high risk.

4 Waterfall doesn't really work that well in an organisation where there are limited

resources and the work that is being carried out is not that well understood initially. For

example, it may take months to get to a point where a build is ready to be tested. If the

test team is limited in capacity (i.e. has to service many projects) then you risk getting

stuck in a loop of extended delays if the builds contain several bugs and need to

undergo repeated testing iterations. You eventually reach a point where months have

gone by and nothing has been released. This doesn't sit well with management.

Agile probably wouldn't work so well in an environment where the team wasn't

experienced / technically capable and the management wasn't willing to support the


206

methodology. The team needs to be willing to take on the challenge of learning to

follow agile but in doing so they need to have enough technical skills to confidently

produce useful deliverables every iteration with limited upfront analysis. Additionally,

if management doesn't support the methodology then you risk being stranded/isolated if

an issue arises that needs to be

5 SCRUM - when strong management support and team commitment is absent. Waterfall

- when requirements gathering / signoff is ineffective.

6 I believe that *how* a method is used is just as important as when it is used.

7 When multiple people are delivering business requirements and they can’t agree.

Table 22: Agile – Question 50 responses

Incremental projects

Project

No


1 A stand-alone waterfall life-cycle is probably a poor choice generally, however using

the waterfall approach with single iterations of an incremental/iterative model will

reduce risk, hence reduce cost and schedule.

2 Incremental works, but our implementation of it fails to enforce any process. We are

really just hacking code – but call it feature driven development. I would definitely like

some more structure and deadlines.

3

4 For any project that is not going well, anything but Waterfall should not be used. Most

reach for the “quick” methodologies, but Waterfall is the best for getting an off-the-

rails project back on track

5

6 What an open question! I think methodologies should be used where appropriate to the

quality of product required, the personnel involved and the business requirements. E.g.

A waterfall approach may make sense for a mission critical system if the timeframes


207

are flexible, the domain is familiar and the cost of software failure is high. Whereas if

the product is experimental an incremental/iterative approach may be more appropriate.

Table 29: Incremental – Question 50 responses

Iterative projects

Project

No


1 I do not think there are conditions in which it should be used. Rather this organisation should be ensuring that methodologies are followed and applied.

Table 35: Iterative – Question 50 responses

Spiral projects

Project

No


1 When the customer cannot understand the needs of the development methodology,

there maybe a need to change the chosen methodology

2 Large projects, most design depends on the experience of the designer

3 All methods can put up a justifiable case as to why they should be used for the next

project as each methodology brings strong benefits to both the development team and

customer. However, we all strive for the Holy Grail of SE process which does not

impose unnecessary burden and supports developing for not only for the current

customer but provides consideration for design goals which may not be required for the

current project but provides a flexible architecture that can sole our next customer's

problem.

That said, I would not recommend the Waterfall method if project requirements/goals

are not mature.

Table 41: Spiral – Question 50 responses


208

Appendix I

Agile methods roles

1. XP, there are many typical roles in the XP process, as stated by [5] and [6]:

1.1 Tracker (is responsible for tracking time estimations, progress and probable risks)

1.2 Customer (is responsible for writing story cards, giving priorities to these story cards

and writing and performing functional tests)

1.3 Programmer (estimates the required efforts for each story card, writes tests and code

that fulfils these tests)

1.4 Coach (supervises the whole process in order to follow the typical XP process)

1.5 Tester (helps to write functional tests and runs them)

1.6 Doomsayer (identifies risks and deviations to plans)

1.7 Manager (makes decisions with the help of the other team members), and

1.8 Consultant (is an external team member who supports the team in solving problems).

2. Scrum, Typical Scrum roles as indicated by [5] and [8]:

2.1 Scrum Master (is responsible for the correct implementation of the Scrum process),

2.2 Scrum Team (self-organising develops the product during each sprint),

2.3 Product Owner (is responsible for the project and its outcome; equivalent to a project

manager), and

2.4 Customer

3. ASD roles as stated by [5]:

3.1 Executive sponsor (has the whole project responsibility)

3.2 Facilitator (organizes the JAD sessions)

3.3 Scribe (makes notes during the JAD sessions)

3.4 Project Manager

3.5 Customer and

3.6 Developer

4. Rational Unified Process (RUP)


209

The roles of RUP as stated by [5]:

4.1 Developer

4.2 Designer

4.3 Architect

4.4 Tester

4.5 Project Manager

4.6 Customer

4.7 Business process analyst (manages the definition of business use-case model)

4.8 Business designer (manages one single business use case)

4.9 Business model reviewer (reviews the work of the business process analyst and the

business designer)

4.10 Course developer (creates course material as e.g. manuals for the users)

4.11 Toolsmith (develops in-house tools to support the development)

Appendix J

Questions support analysis

A B C D E F G H I J K


210

Q26 How many requests have been received for adding new requirements or additional functionality since the project started?

Q27 How many requests for corrections/changes to existing functional requirements have been received since the project started?

Q28 How many requests for changes to existing technology requirements have been received since the project started?

Q29 How many requests for changes to other project factors have been received since the project started?

Table 12: Questions – rate of change in project by requests

A B C D E F G H I J K


211

Appendix K

Participants comments

I thought a lot about if I should include or not include participants comments in the report, and I decided to include them, but without identifying any participant.

Comments provided as given/received from participants. I will not comment on them, but if I add any text it will be in blue colour.

It was quite hard to choose a lifecycle, as nothing formal is in place, so even though I'd like to say "chaos"", I think what I've chosen is more fair. I'd like to add a personal note though:

The highest quality software I've seen is still CEA.

The more I move around and talk with people, the more I think the problems in the software industry are psychological and sociological, rather than anything else.

Follow participant stated the reason why people not using methodology:

- People don't care, they just want to tinker with code.

- The benefit of working in a better way takes effort to change and the benefits are intangible

(abstract, and potential future benefits).

- IT people are largely meek and apolitical, yet passive-aggressive. If someone wants a

requirements change they don't care about but is potentially harmful they'll just do it so they aren't

seen as a trouble maker, or they'll argue but get overruled by a stronger personality. If someone

wants a process change that affects the way they work they'll agree but sabotage it.

- Everyone else works that way and no one wants to stick their neck out and change.

- People think what they're doing IS the latest and greatest way, so they don't need to change,

they're already on the forefront.

- People think their domain is special so the industry standard techniques don't apply to them.


212

Appendix L

Systematic approach

Project objective

Research Method

Survey objectives

Questionnaire design Questionnaire construction implementation

Analysis

Collected data

Report


213

References

1. DOD-STD-2167A (Department of Defence Standard 2167A), titled "Defense Systems Software Development", http://en.wikipedia.org/wiki/DOD-STD-2167A

2. B. W. Boehm, A Spiral Model of Software Development and Enhancement, pp. 60–72.

3. http://www.jimhighsmith.com/articles/messy.htm, 27.04.2009

4. Scott Plamondon, working Smarter, not harder – An interview with Kent Beck, IBM developerWorker, 2003, http://www-106.ibm.com/developerworks/java/library/j-beck/?ca=dgr-lnxw16KentBeck, 20.04.2009

5. Pekka Abrahamsson, Outi Salo, Jussi Ronkainen and Juhani Warsta, “Agile software development methods – review and analysis”, (2002)

6. Ward Cunningham, Extreme Programming Roadmap, http://c2.com/cgi/wiki?ExtremeProgrammingRoadmap , 21.04.2009

7. Jeff Sutherland, Jeff Sutherland’s SCRUM Log, http://jeffsutherland.org/scrum/ , 22.04.2009

8. ADM (ed.), Scrum, http://www.controlchaos.com , 25.04.2009

9. DSDM Consortium, DSDM, http://www.dsdm.org , 25.04.2009

10. Ted R. Compton, “Minimizing waste with RAD”, in: Strategic Finance, June 2002, pp. 50-53

11. Highsmith, J. “Messy, Exiting, and Anxiety-Ridden: Adaptive Software Development”, in: American Programmer, Volume X, No. 1, 1997, http://www.jimhighsmith.com/articles/messy.htm , 27.04.2009

12. http://www.devx.com/architect/Article/32836/0/page/2 , 27.04.2009

13. Claes Wohlin, Per Runesson, Martin Höst, Magnus C. Ohlsson, Björn Regnell, Anders Wesslen: "Empirical Strategies", excerpt from "Experimentation in Software Engineering - An introduction", pp. 7-15, Kluwer Academic Publishers, 2000. http://books.google.com.au/books?id=nG2UShV0wAEC&pg=PA8&lpg=PA8&dq=Claes+Wohlin,+Empirical+Strategies&source=bl&ots=9Fi8QU7iXl&sig=9IxYvSoHMLziwlWHGvx6bN7o9tw&hl=en&ei=Rr_KS7auMMyOkQXr3aTJBA&sa=X&oi=book_result&ct=result&resnum=4&ved=0CBUQ6AEwAw#v=onepage&q=Claes%20Wohlin%2C%20Empirical%20Strategies&f=false , Online access on 22/06/2009.

14. Fitzgerald, B. (1998). An empirically-grounded framework for the information systems development process, Proceedings of the International Conference on Information Systems, (pp. 100-115).

15. Lytinnen, K. & Rose, G. M. (2003). The disruptive nature of information technology innovations; The case of internet computing in systems development organisations, 555-590.


214

16. Cameron, K. S. & Quinn, R. E. (1999). Diagnosing and changing organisational culture. online access 18.05.2009. http://books.google.com.au/books?id=blRwWniTsUAC&printsec=frontcover&dq=Diagnosing+and+Changing+Organizational+Culture+by+Cameron+and+Quinn+1999&source=gbs_summary_r&cad=0#PPP1,M1

17. Williams, Kerbs and Layman, 2004. online access 15.05.2009. ftp://ftp.ncsu.edu/pub/unity/lockers/ftp/csc_anon/tech/2004/TR-2004-18.pdf

18. B.Boehm & Turner, 2004: Cockburn, 2002. online access 15.5.2009. http://www.informit.com/articles/article.aspx?p=26057

19. Subhas C. Misra, Vinod Kumar, and Uma Kumar. Online access 22.05.2009. http://ww1.ucmss.com/books/LFS/CSREA2006/SER5088.pdf

20. B.Boehm & Turner, 2004: Cockburn, 2002. http://books.google.com.au/books?id=ttaMIFv8bv8C&pg=PA568&lpg=PA568&dq=B.Boehm+%26+Turner,+2004&source=bl&ots=yWkmQ5oPJl&sig=taQr5edFcQpg8DpwAyd0Tw-B3ig&hl=en&ei=hyoZStKUBpHy6gPZwrCvDg&sa=X&oi=book_result&ct=result&resnum=1, online access on 22.06.2009

21. Claes Wohlin, Per Runesson, Martin Host, Magnus C. Ohlsson, Bjorn Regnell, Anders Wesslen: "Empirical Strategies", excerpt from "Experimentation in Software Engineering - An introduction", pp. 7-15, Kluwer Academic Publishers, 2000. http://books.google.com.au/books?id=nG2UShV0wAEC&pg=PA8&lpg=PA8&dq=Claes+Wohlin,+Empirical+Strategies&source=bl&ots=9Fi8QU7iXl&sig=9IxYvSoHMLziwlWHGvx6bN7o9tw&hl=en&ei=Rr_KS7auMMyOkQXr3aTJBA&sa=X&oi=book_result&ct=result&resnum=4&ved=0CBUQ6AEwAw#v=onepage&q=Claes%20Wohlin%2C%20Empirical%20Strategies&f=false, Online access on 22.06.2009

22. Software Development Research Foundation. On line access 24.04.2010 http://www.sdresearch.org/

23. Glass, R. L. (2004). Matching methodology to problem domain. Communications of the ACM, p (19).

24. Cockburn, A. (2002). Agile Software Development.

Lindvall, M. Agile Software Development in Large Organisation, online access on 18.12.2009 http://www.agile-itea.org/public/papers/AGILE_Large_Organizations.pdf

26. Australian Bureau of Statistics, 1321.0 - Small Business in Australia, 2001, Released at 11:30 AM (CANBERRA TIME) 23/10/2002, Online access on 28.04.2010, http://www.abs.gov.au/ausstats/[email protected]/ProductsbyTopic/97452F3932F44031CA256C5B00027F19?OpenDocument

27. Gerald Kotonya, Ian Sommerville, “Requirements Engineering”, John Wiley & Sons, Chichester UK, 1998.


215

28. Crinnion, J. 1992. The evolutionary development of business systems, p (22- 27).

29. Roberts, T.L, Leigh, W., Purvis, R, L. (2000). Perceptions on stakeholder involvement in the implementation of system development methodologies, p(75-88)

30. Hevner, A,. Collins, R. W., & Garfield, M. J. (2002). Product and project challenges in electronic commerce software development, (pp.11-25)

31. Butler, T., & Fitzgerald, B. (2001). The relationship between user participation and the management of change surrounding the development of information systems, (pp.10-30)

32. Wasserman, A. I., Freeman, P., & P., & Porcella, M. (1983). Characteristics of software development methodologies

33. Blum, Bruce I., Software Engineering: A Holistic View, (1992), (pp.30).

34. Agile software development: online access 25.03.2010 http://www.scribd.com/doc/3281936/Agile-software-development

35. Schwaber, K,. & Beedle, M. (2002). Agile Software Development with Scrum.

36. Highsmith, J. (2003). Agile Software Development.

37. Bunse, C. Feldmann, R. L., & Dorr, J. (2004). Agile methods in software engineering education.

38. Beck, K. (1999). Embracing change with Extreme Programming. (pp.68-78)

39. Beck, K. (2000). Extreme Programming explained: Embrace change.(pp.20-100)

40. McKeen, J. D,. Guimaraes, T., & Wetherbe, J. C. (1994). The relationship between user participation and user satisfaction, p(425-452).

41. How Google Got Its New Look, online access 28.05.2010 http://www.businessweek.com/magazine/content/10_20/b4178000295757.htm

42. Software development methodology, online access 28.05.2010 http://www.hyperthot.com/pm_sdm.htm

43. Software development methodology today, online access 23.05.2009 http://ptgmedia.pearsoncmg.com/images/0131872508/samplechapter/0131872508_ch01

44. DOD-STD-2167, online access 23.04.2009 http://www.product-lifecycle-management.com/download/DOD-STD-2167A.pdf

This Report is part of the requirement for the Master degree in...

Documents

Transcript of This Report is part of the requirement for the Master degree in...